Posted Python涓枃绀惧尯
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了相关的知识，希望对你有一定的参考价值。
銆孈Author: Elvis銆?/span>
娆㈣繋鏉ュ埌 NLP 鏃朵簨绠€鎶ワ紒鍏ㄦ枃杈冮暱锛屽缓璁敹钘忋€?/p>
-
1銆丳ublications 馃摍 1.1 Turing-NLG from Mircosoft
1.2 Neural based Dependency Parsing
1.3 End-to-end Cloud-based Information Extraction with BERT
1.4 Question Answering Benchmark
1.5 Radioactive data: tracing through training
1.6 REALM: Retrieval-Augmented Language Model Pre-Training
1.7 ERNIE-GEN from Baidu
-
2銆丆reativity and Society 馃帹 -
2.1 鍏佽鍦ㄧ瀛︿細璁笂杩涜杩滅▼璁烘枃鍜屾捣鎶ュ睍绀? -
2.2 Abstraction and Reasoning Challenge -
2.3 ML and NLP Publications in 2019 -
2.4 Growing Neural Cellular Automata -
2.5 Transformer attention鍙鍖? -
2.6 SketchTransfer -
2.7 The future of NLP in Python
3銆乀ools and Datasets 鈿欙笍
3.1 DeepSpeed + ZeRO
3.2 涓€涓敤浜庤繘琛屽揩閫熸湁鏁堢殑3D娣卞害瀛︿範鐮旂┒鐨勫簱
3.3 绠＄悊浣犵殑ML椤圭洰閰嶇疆
3.4 璐濆彾鏂綉缁滃洜鏋滄帹鐞嗗伐鍏峰寘
3.5 TyDi闂瓟锛氬璇█闂瓟鍩哄噯
3.6 Question Answering for Node.js
3.7 鐏垫椿鑰屽己澶х殑NLP宸ュ叿鍖?br>
4銆丒thics in AI 馃毃
-
4.1 璇嗗埆鏂囨湰涓殑subjective bias -
4.2 Artificial Intelligence, Values and Alignment -
4.3 鍏充簬瀹℃牳AI绯荤粺 5銆丄rticles and Blog posts 鉁嶏笍
-
5.1 鐢ㄤ簬NLP绯荤粺鐨勬ā鍨嬭捀棣? -
5.2 鍗曡瘝鐨勪笂涓嬫枃琛ㄧず -
5.3 绁炵粡缃戠粶涓殑绋€鐤忔€? -
5.4 璁粌浣犺嚜宸辩殑璇█妯″瀷 -
5.5 鍒嗚瘝鍣═okenizer 6銆丒ducation 馃帗
-
6.1 闃垮鏂壒涓硅嚜鐢卞ぇ瀛︽満鍣ㄥ涔犺绋? -
6.2 鏈哄櫒瀛︿範鏁板璧勬簮 -
6.3 娣卞害瀛︿範鍏ラ棬 -
6.4 Pytorch娣卞害瀛︿範 -
6.5 Missing Semester of Your CS -
6.6 娣卞害瀛︿範杩涢樁 -
7銆丯oteworthy Mentions 猸愶笍
1銆丳ublications 馃摍
1.1 Turing-NLG: A 17-billion-parameter language model by Microsoft
鍥剧伒鑷劧璇█鐢熸垚锛圱-NLG锛?sup class="mq-67">[1]鏄敱 Microsoft AI 鐮旂┒浜哄憳鎻愬嚭鐨?170 浜垮弬鏁拌瑷€妯″瀷銆傞櫎浜嗘槸杩勪粖涓烘鏈€澶х殑宸茬煡璇█妯″瀷锛堝涓嬪浘鎵€绀猴級涔嬪锛孴-NLG 鏄熀浜?78 灞?Transformer 鐨勮瑷€妯″瀷锛屽叾鍦?WikiText-103 涓婄殑鍥版儜搴︽€ц兘浼樹簬涔嬪墠鐨勬渶鏂版妧鏈垚鏋滐紙鐢盢VIDIA Megatron-LM[2]鎸佹湁锛?銆俆-NLG 鍦ㄥ悇绉嶄换鍔★紙渚嬪闂鍥炵瓟鍜屾娊璞℃憳瑕侊級涓婅繘琛屼簡娴嬭瘯锛屽悓鏃跺垎鍒樉绀轰簡妯″瀷鐨勫ソ澶勶紝渚嬪闆剁畝鐭棶棰樺姛鑳藉拰鏈€灏忓寲鐩戠潱銆傛澶栵紝璇ユā鍨嬪緱鐩婁簬 DeepSpeed 搴擄紙涓?PyTorch 鍏煎锛夊拰 ZeRO 浼樺寲鍣紝杩欎袱鑰呬篃浼氬湪鏈湡绠€鎶ヤ腑鍏蜂綋浠嬬粛銆?/p>
璇█妯″瀷鍙傛暟澶у皬
1.2 Neural based Dependency Parsing
Miryam de Lhoneux 鍏竷浜嗗叾鍗氬＋瀛︿綅璁烘枃鈥淟inguistically Informed Neural Dependency Parsing for Typologically Diverse Languages[3]鈥濄€傝繖椤瑰伐浣滄槸鍏充簬浣跨敤绁炵粡瀛︽柟娉曚互绫诲瀷澶氭牱鐨勮瑷€锛堝嵆浠ョ粨鏋勪笂涓嶅悓鐨勬柟寮忔瀯閫犲拰琛ㄨ揪鍚箟鐨勮瑷€锛夎繘琛屼緷璧栧叧绯昏В鏋?sup class="mq-76">[4]銆傝鏂囨寚鍑?RNN 鍜岄€掑綊灞傚彲鑳芥湁鍔╀簬鍚堝苟鍒拌В鏋愬櫒涓紝鍥犱负瀹冧滑鏈夊姪浜庡憡鐭ュ叿鏈夎В鏋愭墍闇€鐨勯噸瑕佽瑷€鐭ヨ瘑鐨勬ā鍨嬨€傛洿澶?ideas 鍖呮嫭浣跨敤澶氳瑷€瑙ｆ瀽鍜屽弬鏁板叡浜瓥鐣ユ潵瑙ｆ瀽鐩稿叧鍜屼笉鐩稿叧璇█銆?/p>
1.3 End-to-end Cloud-based Information Extraction with BERT
鐮旂┒浜哄憳鍙戣〃浜嗕竴绡囪鏂?Rapid Adaptation of BERT for Information Extraction on Domain-Specific Business Documents"[5]锛屾弿杩颁簡 BERT 绛?Transformer 妯″瀷濡備綍甯姪鐗瑰畾棰嗗煙鐨勪笟鍔℃枃妗ｏ紙渚嬪鐩戠鏂囦欢鍜岀墿涓氱璧佸崗璁級涓殑绔埌绔俊鎭彁鍙栥€傝繖绉嶇被鍨嬬殑宸ヤ綔涓嶄粎鍙互甯姪浼樺寲涓氬姟杩愯惀锛岃€屼笖杩樻樉绀轰簡鍩轰簬 BERT 鐨勬ā鍨嬪湪甯︽敞閲婃暟鎹緢灏戠殑鎯呭喌涓嬬殑閫傜敤鎬у拰鏈夋晥鎬с€傚悓鏃舵彁鍑哄苟璁ㄨ浜嗗湪浜戜笂杩愯鐨勫簲鐢ㄧ▼搴忓強鍏跺疄鐜扮粏鑺傦紙璇峰弬瑙佷笅鍥撅級銆?img data-ratio="0.54125" src="/img?url=https://mmbiz.qpic.cn/mmbiz_png/DHibuUfpZvQc59wGKwjDC7oibrKLghA7He52un7bFpIP9wDNvA4W4gvj3bibRdhxFFxGibT2ovJroLh8rjSaWn5LLQ/640?wx_fmt=png" data-type="png" data-w="800" _width="100%" class="mq-81" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?>
1.4 Question Answering Benchmark
Wolfson 绛夊彂甯冧簡涓€涓猶uestion understanding benchmark[6]锛屼互鍙婁竴绉嶇敤浜庡垎瑙ｈ绠楅€傚綋绛旀鎵€蹇呴渶鐨勯棶棰樼殑鏂规硶銆備粬浠埄鐢ㄤ紬鍖呮潵娉ㄩ噴鍒嗚В闂鎵€闇€鐨勫繀瑕佹楠わ紝 涓轰簡灞曠ず璇ユ柟娉曠殑鍙鎬у拰閫傜敤鎬э紝浠栦滑鏀硅繘浜嗕娇鐢?HotPotQA 鏁版嵁闆嗙殑寮€鏀惧煙闂瓟銆?/p>
涓嶅悓鏉ユ簮鐨勯棶棰樺叿鏈夌浉浼肩殑缁勬垚缁撴瀯銆?
1.5 Radioactive data: tracing through training
Facebook AI 鐮旂┒浜哄憳鏈€杩戝彂琛ㄤ簡涓€椤规湁瓒ｇ殑宸ヤ綔[7]锛屾棬鍦ㄦ爣璁板浘鍍忥紙绉颁负銆宺adioactive data銆?/span>锛夛紝浠ラ獙璇佽鐗瑰畾鏁版嵁闆嗘槸鍚︾敤浜庤缁?ML 妯″瀷銆備粬浠彂鐜帮紝鍙互浣跨敤宸у鐨勬爣璁板皢鐗瑰緛绉诲悜鏌愪釜鏂瑰悜锛屽嵆浣垮彧鏈?1锛呯殑璁粌鏁版嵁鏄?radioactiv锛屾ā鍨嬩篃鍙互浣跨敤璇ユ爣璁板府鍔╂娴嬧€榬adioactive data鈥欑殑浣跨敤鎯呭喌銆傝繖鏋佸叿鎸戞垬鎬э紝鍥犱负鏁版嵁涓殑浠讳綍鏇存敼閮藉彲鑳戒細闄嶄綆妯″瀷鐨勫噯纭€с€備綔鑰呰锛岃繖椤瑰伐浣滃彲浠モ€滃府鍔╃爺绌朵汉鍛樺拰宸ョ▼甯堣窡韪敤浜庤缁冩ā鍨嬬殑鏁版嵁闆嗭紝浠ヤ究浠栦滑鍙互鏇村ソ鍦颁簡瑙ｅ悇绉嶆暟鎹泦濡備綍褰卞搷涓嶅悓绁炵粡缃戠粶鐨勬€ц兘鈥濓紝鍦ㄥ叧閿换鍔?ML 搴旂敤绋嬪簭涓紝杩欎技涔庢槸涓€绉嶉噸瑕佺殑鏂规硶銆傚鏋滄劅鍏磋叮鍙互鏌ョ湅瀹屾暣璁烘枃[8]浜嗚В鍏蜂綋淇℃伅銆?/p>
REALM 鏄熀浜庣缁忕殑澶ц妯℃绱㈡柟娉曪紝璇ユ柟娉曞埄鐢ㄦ枃鏈煡璇嗙殑璇枡搴撲互鏃犵洃鐫ｇ殑鏂瑰紡棰勮缁冭瑷€妯″瀷銆傝鏂规硶鐨勪富瑕佺洰鐨勬槸浠ヤ竴绉嶆洿鍙В閲婄殑鏂瑰紡鎹曡幏鐭ヨ瘑锛屽叿浣撹€岃█鏄埄鐢ㄤ笘鐣岀煡璇嗗妯″瀷杩涜璁粌鍜岄娴嬨€備娇鐢?REALM 澶勭悊鍜岃瘎浼扮殑浠诲姟鍖呮嫭寮€鏀惧煙闂瓟鍩哄噯锛?闄や簡鎻愰珮妯″瀷鐨勫噯纭€у锛屽叾浠栧ソ澶勮繕鍖呮嫭妯″潡鍖栧拰鍙В閲婃€х粍浠躲€?img data-ratio="0.9700598802395209" src="/img?url=https://mmbiz.qpic.cn/mmbiz_png/DHibuUfpZvQc59wGKwjDC7oibrKLghA7HeF5TuETib5qSawia7d2SbZzY63sqYbnIwPTAoGKdKXXPL9GNficTHTuAZw/640?wx_fmt=png" data-type="png" data-w="501" _width="100%" class="mq-98" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?> 鐧惧害鐮旂┒鑰呮彁鍑轰簡涓€绉嶅寮虹殑澶氭祦搴忓垪锛岀敤浜庡簭鍒楅璁粌鍜屽井璋冩鏋讹紝鍚嶄负ERNIE-GEN[9]锛岃妗嗘灦閫氳繃濉厖鐢熸垚鏈哄埗鍜屽櫔澹版劅鐭ョ敓鎴愭柟娉曞讥鍚堜簡璁粌鍜屾帹鐞嗕箣闂寸殑宸紓銆備负浜嗕娇鐢熸垚鏇存帴杩戜汉绫荤殑涔﹀啓妯″紡锛屾妗嗘灦寮曞叆浜嗛€愯法鐢熸垚娴侊紝璇ユ祦瀵规ā鍨嬭繘琛岃缁冧互杩炵画棰勬祴璇箟涓婂畬鏁寸殑璺ㄨ窛锛岃€屼笉鏄€愬瓧棰勬祴銆備笌鐜版湁鐨勯璁粌鏂规硶涓嶅悓锛孍RNIE-GEN缁撳悎浜嗗绮掑害鐩爣閲囨牱鏉ユ瀯閫犻璁粌鏁版嵁锛屼粠鑰屽寮轰簡缂栫爜鍣ㄥ拰瑙ｇ爜鍣ㄤ箣闂寸殑鐩稿叧鎬с€傚疄楠岀粨鏋滆〃鏄庯紝ERNIE-GEN鍙互鍦ㄤ竴绯诲垪璇█鐢熸垚浠诲姟涓婁互杈冨皯鐨勯璁粌鏁版嵁鍜屽弬鏁版潵鑾峰緱鏈€鏂扮殑缁撴灉銆?/p>
杩囧幓涓€鍛ㄦ湁璇锋効涔︽暎鍙戯紝浠ヤ究鍦ㄤ笌 ML 鐩稿叧鐨勭瀛︿細璁笂杩涜杩滅▼璁烘枃鍜屾捣鎶ュ睍绀猴紝鍙互鍦╟hange.org[10]涓婇槄璇绘湁鍏冲畠鐨勬洿澶氫俊鎭€傛繁搴﹀涔犵殑鍏堥┍ Yoshua Bengio 浼间箮鍦ㄥ€″浜轰滑鍘荤缃茶鎰夸功锛?骞跺湪浠栫殑鏂板崥瀹?sup class="mq-109">[11]涓槓鏄庝簡杩欎竴鐐广€?/p>
Fran莽oisChollet 鏈€杩戝彂甯冧簡涓€涓?Kaggle 绔炶禌锛屼粬鍙戝竷浜嗘娊璞℃帹鐞嗚鏂欏簱锛圓RC锛?sup class="mq-113">[12]锛屾棬鍦ㄩ紦鍔辩敤鎴峰垱寤哄彲浠ヨВ鍐充粠鏈帴瑙﹁繃鐨勬帹鐞嗕换鍔＄殑 AI 绯荤粺銆傚笇鏈涜兘澶熷紑濮嬫瀯寤烘洿寮哄ぇ鐨?AI 绯荤粺锛屼粠鑰岃兘澶熸洿濂斤紝蹇€熷湴鑷瑙ｅ喅鏂伴棶棰橈紝杩欏彲鑳芥湁鍔╀簬瑙ｅ喅鏇村叿鎸戞垬鎬х殑鐜板疄搴旂敤锛屼緥濡傛敼鍠勫湪鏋佺鍜屽鏍峰寲鐜涓繍琛岀殑鑷姩椹鹃┒姹借溅 銆?/p>
Marek Rei 鍙戝竷鏈哄櫒瀛︿範鍜?NLP 棰嗗煙 2019 骞村嚭鐗堟暟鎹粺璁?sup class="mq-117">[13]锛屽垎鏋愪腑鍖呮嫭鐨勪細璁槸 ACL锛孍MNLP锛孨AACL锛孍ACL锛孋OLING锛孴ACL锛孋L锛孋oNLL锛孨eurIPS锛孖CML锛孖CLR 鍜?AAAI銆?/p>
Hendrik Strobelt 寮€婧愰」鐩樉绀轰簡濡備綍浣跨敤 Hugging Face 搴撳拰 d3.js 閫氳繃 Web 搴旂敤绋嬪簭蹇€熸瀯寤虹畝鍗曠殑浜や簰寮?Transformer 娉ㄦ剰鍙鍖?sup class="mq-127">[15]銆?img data-ratio="0.7666666666666667" src="/img?url=https://mmbiz.qpic.cn/mmbiz_gif/DHibuUfpZvQc59wGKwjDC7oibrKLghA7He7ShzkIicOqO0WVuYh5icDBVBINAQLUFxDVy6PGtQhNAGHSqIrp1fw76g/640?wx_fmt=gif" data-type="gif" data-w="600" _width="100%" class="mq-128" data-order="0" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?> SketchTransfer[16]鎻愬嚭浜嗕竴椤规柊浠诲姟锛屼互娴嬭瘯娣卞害绁炵粡缃戠粶鍦ㄥ瓨鍦?涓嶅瓨鍦ㄧ粏鑺傛椂鏀寔涓嶅彉鎬х殑鑳藉姏銆傞暱鏈熶互鏉ヤ竴鐩翠簤璁虹殑鏄紝娣卞害缃戠粶鏃犳硶鎺ㄥ箍鍒拌缁冩湡闂村皻鏈彂鐜扮殑鍙樺寲锛屼汉绫诲彲浠ョ浉瀵硅交鏉惧湴瀹屾垚鏌愪簺浜嬫儏锛屼緥濡傚湪瑙傜湅鍔ㄧ敾鐗囨椂澶勭悊涓㈠け鐨勮瑙夌粏鑺傘€傛湰鏂囪璁哄苟鍙戝竷浜嗕竴涓暟鎹泦锛屼互閫氳繃鎻愪緵鏈爣璁扮殑鑽夊浘鍥惧儚鍜屾爣璁扮殑鐪熷疄鍥惧儚绀轰緥鏉ュ府鍔╃爺绌朵汉鍛樹粩缁嗙爺绌垛€滅粏鑺備笉鍙樻€р€濋棶棰樸€?img data-ratio="1.1105769230769231" src="/img?url=https://mmbiz.qpic.cn/mmbiz_png/DHibuUfpZvQc59wGKwjDC7oibrKLghA7HeqlBRgRau1icHCaSBpl9UcOnpWlNytAqw9TbcM0e19nortk2HIoib91SA/640?wx_fmt=png" data-type="png" data-w="416" _width="100%" class="mq-133" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?> Ines Montani鍏充簬Python NLP灞曟湜[17]鐨勪竴缁勯潪甯告湁鎰忔€濈殑PPT浠嬬粛銆?/p>
Microsoft 寮€婧愪簡涓€涓О涓?DeepSpeed 鐨勮缁冧紭鍖栧簱锛岃搴撲笌 PyTorch 鍏煎锛屽苟涓斿彲浠ヨ缁?1000 浜夸釜鍙傛暟鐨勬ā鍨嬨€傝搴撲晶閲嶄簬璁粌妯″瀷鐨勫洓涓噸瑕佹柟闈細瑙勬ā锛岄€熷害锛屾垚鏈拰鍙敤鎬с€侱eepSpeed 涓?ZeRO[18]涓€璧峰彂甯冿紝ZeRO 鏄竴绉嶅唴瀛樹紭鍖栨妧鏈紝鍙湪褰撳墠 GPU 鎶€鏈腑瀹炵幇澶ц妯″垎甯冨紡娣卞害瀛︿範锛屽悓鏃跺皢鍚炲悙閲忔彁楂樺埌褰撳墠鏈€浣崇郴缁熺殑涓夊埌浜斿€嶃€俍eRO 鍏佽璁粌浠绘剰澶у皬鐨勬ā鍨嬶紝杩欎簺妯″瀷鍙互鏍规嵁鍏变韩妯″瀷鐘舵€侀€傚悎鑱氬悎鐨勫彲鐢ㄥ唴瀛樸€?img data-ratio="0.5628571428571428" src="/img?url=https://mmbiz.qpic.cn/mmbiz_gif/DHibuUfpZvQc59wGKwjDC7oibrKLghA7HefziaIu5dZERo5iaOEv1XKfAsZDbAftUXLetgOLv6EWmr3rIzMZ99KZ0A/640?wx_fmt=gif" data-type="gif" data-w="1400" _width="100%" class="mq-144" data-order="1" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?> PyTorch3D[19]鏄敤浜庡熀浜?3D 鐨勬繁搴﹀涔犵爺绌剁殑寮€婧愬伐鍏峰寘銆傝繖涓?PyTorch 搴撴棬鍦ㄥ府鍔╂繁搴﹀涔犵郴缁熶腑鐨?3D 鏁版嵁鏀寔鍜岀悊瑙ｏ紝璇ュ簱鐢卞父鐢?3D 杩愮畻绗﹀拰鎹熷け鍑芥暟鐨勫揩閫熶紭鍖栧疄鐜扮粍鎴愶紝 瀹冭繕甯︽湁妯″潡鍖栫殑鍙尯鍒嗘覆鏌撳櫒锛屽彲甯姪杩涜澶嶆潅 3D 杈撳叆鐨勭爺绌跺苟鏀寔楂樿川閲?3D 棰勬祴銆?img data-ratio="0.5792682926829268" src="/img?url=https://mmbiz.qpic.cn/mmbiz_png/DHibuUfpZvQc59wGKwjDC7oibrKLghA7HeUOZ1y45YCwCZrQEOXWAPLNl4hlXubu6tt0SZNrq8YicmyKMu2PicL0aA/640?wx_fmt=png" data-type="png" data-w="656" _width="100%" class="mq-149" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?> Hydra[20]鏄熀浜?Python 鐨勯厤缃伐鍏凤紝鐢ㄤ簬鏇存湁鏁堝湴绠＄悊澶嶆潅鐨?ML 椤圭洰銆傚畠鏃ㄥ湪閫氳繃涓?ML 椤圭洰鎻愪緵鍔熻兘鐨勯厤缃噸鐢ㄦ潵甯姪 PyTorch 鐮旂┒浜哄憳銆傚畠鎻愪緵鐨勪富瑕佸ソ澶勬槸瀹冨厑璁哥▼搴忓憳鍍忕紪鍐欎唬鐮佷竴鏍风紪鍐欓厤缃紝杩欐剰鍛崇潃鍙互杞绘澗鍦拌鐩栭厤缃枃浠躲€侶ydra 杩樺彲浠ュ府鍔╄嚜鍔ㄧ鐞?ML 椤圭洰杈撳嚭鐨勫伐浣滅洰褰曪紝杩欏湪闇€瑕佷繚瀛樺拰璁块棶澶氫釜浣滀笟鐨勫涓疄楠岀粨鏋滄椂闈炲父鏈夌敤銆?/p>
CausalNex[21]鏄敤浜庘€滀笌璐濆彾鏂綉缁滆繘琛屽洜鏋滄帹鐞嗏€濈殑宸ュ叿鍖呫€傝宸ュ叿鏃ㄥ湪灏嗘満鍣ㄥ涔犲拰鍥犳灉鎺ㄧ悊鐩哥粨鍚堬紝浠ュ彂鐜版暟鎹腑鐨勭粨鏋勫叧绯汇€備綔鑰呰繕鍑嗗浜嗗叧浜庝娇鐢?Python 搴撴帹鏂礉鍙舵柉缃戠粶鍘熷洜鍜屽師鍥犵殑鍏ラ棬鎸囧崡銆?img data-ratio="0.5458937198067633" src="/img?url=https://mmbiz.qpic.cn/mmbiz_png/DHibuUfpZvQc59wGKwjDC7oibrKLghA7HeaV2ZEicOkvwAeqUbZntAHIPLSfTSBZTxborEEJ6icLLvaX2nJ2LPYHCw/640?wx_fmt=png" data-type="png" data-w="621" _width="100%" class="mq-158" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?> Google AI 鍙戝竷浜員yDi QA[22]锛屽畠鏄竴绉嶅璇█鐨勬暟鎹泦锛屽彲浠ラ紦鍔辩爺绌朵汉鍛樺鏇村绫诲瀷澶氭牱鐨勮瑷€杩涜闂瓟锛岃繖浜涜瑷€浠ヤ笉鍚岀殑鏂瑰紡鏋勫缓鍜岃〃杈惧惈涔夈€傝繖鏍峰仛鐨勭洰鐨勬槸婵€鍔辩爺绌朵汉鍛樺湪绫诲瀷涓婅窛绂婚仴杩滅殑璇█锛堜緥濡傞樋鎷変集璇紝瀛熷姞鎷夎锛岄煩璇紝淇勮锛屾嘲鍗㈠浐璇拰娉拌锛変笂寤虹珛鏇村己澶х殑妯″瀷锛屼互渚垮皢鍏舵帹骞垮埌鏇村绉嶈瑷€銆?img data-ratio="0.3942446043165468" src="/img?url=https://mmbiz.qpic.cn/mmbiz_png/DHibuUfpZvQc59wGKwjDC7oibrKLghA7HejicZH8cFPvuau2APe3JzDCXpBia3bxksUMRSBUyzEcqFqcu9BOfHLouA/640?wx_fmt=png" data-type="png" data-w="695" _width="100%" class="mq-163" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?> Hugging Face 鍙戝竷浜嗗熀浜?DistilBERT 鐨勯棶绛斿簱[23]锛屽苟缁х画浣?NLP 鏇村姞鏄撲簬璁块棶銆傝妯″瀷鍙互浣跨敤 Node.js 鍦ㄧ敓浜т腑杩愯锛屽彧闇€ 3 琛屼唬鐮併€傝妯″瀷鍒╃敤浜嗙敱 Hugging Face 鍜?TensorFlow.js锛堢敤浜庡皢鏈哄櫒瀛︿範妯″瀷涓?Javascript 缁撳悎浣跨敤鐨勬祦琛屽簱锛夋瀯寤虹殑 Tokenizer 鐨勫揩閫熷疄鐜般€?/p>
Forte[24]鏄敤浜庢瀯寤鸿嚜鐒惰瑷€澶勭悊绠￠亾鐨勫伐鍏峰寘锛屽叿鏈夎法浠诲姟浜や簰锛岄€傚簲鎬у己鐨勬暟鎹ā鍨嬫帴鍙ｇ瓑鍔熻兘銆傚畠鎻愪緵浜嗕竴涓互楂樺害鍙粍鍚堢殑鏂瑰紡缁勮鏈€鏂扮殑NLP鍜孧L鎶€鏈殑骞冲彴锛屽寘鎷粠淇℃伅妫€绱紝鑷劧璇█鐞嗚В鍒拌嚜鐒惰瑷€鐢熸垚鐨勫箍娉涗换鍔°€?/p>
璁＄畻绀句細绉戝鐨勭爺绌跺憳 Diyi Yang 璁ㄨ浜?AI 绯荤粺濡備綍甯姪璇嗗埆鏂囨湰淇℃伅涓殑涓昏鍋忚[25]銆傝繖鏄秹鍙?AI 绯荤粺鍜?NLP 鐨勯噸瑕佺爺绌堕鍩燂紝灏ゅ叾鏄綋鎴戜滑璁ㄨ璇稿鏂伴椈鏍囬涔嬬被鐨勬枃鏈獟浣撶殑娑堣垂鏃讹紝寰堝鏄撳皢鍏舵瀯鏋舵垚鍋忓悜娑堣垂鑰咃紝鑰屽疄闄呬笂浠栦滑搴旇杩芥眰鏇村姞瀹㈣銆備粠搴旂敤绋嬪簭鐨勮搴︽潵鐪嬶紝鑷姩璇嗗埆鏂囨湰濯掍綋涓瓨鍦ㄧ殑涓昏鍋忚浠ュ府鍔╂秷璐硅€呮洿鍔犱簡瑙ｄ粬浠鍦ㄦ秷璐圭殑鍐呭鍙樺緱鑷冲叧閲嶈銆傛澶栬繕璁ㄨ浜?AI 濡備綍涔熷彲浠ヤ繚鎸佸亸瑙併€?/p>
浜哄伐鏅鸿兘绯荤粺鐨勫叴璧蜂互鍙婂畠浠浣曚笌浜虹被浠峰€艰淇濇寔涓€鑷存槸娑夊強浜哄伐鏅鸿兘绯荤粺浼︾悊瀛︾殑娲昏穬鐮旂┒棰嗗煙銆侱eepMind[26]鏈€杩戝彂甯冧簡涓€绡囪鏂囷紝娣卞叆鎺㈣浜嗗洿缁?AI 瀵归綈鐨勫摬瀛﹂棶棰樸€傝鎶ュ憡閲嶇偣璁ㄨ浜嗕袱涓儴鍒嗭紝鍗虫妧鏈儴鍒嗭紙鍗冲浣曞浠?AI 浠ｇ悊鑾峰緱鍙潬缁撴灉鐨勫€艰繘琛岀紪鐮侊級鍜岃鑼冩€э紙鍦?AI 涓繘琛岀紪鐮佺殑鍘熷垯鏄纭殑锛変互鍙婂畠浠箣闂寸殑鑱旂郴浠ュ強鍙互纭繚鐨勯儴鍒嗐€傛湰鏂囦富寮犻噰鐢ㄤ竴绉嶅熀浜庡師鍒欑殑 AI 瀵归綈鏂规硶锛屽苟鍦ㄤ俊蹇靛拰瑙傜偣瀛樺湪宸紓鐨勬儏鍐典笅淇濇寔鍏钩寰呴亣銆?/p>
VentureBeat 鎶ュ憡绉帮紝Google 鐮旂┒浜哄憳涓庡叾浠栧皬缁勫悎浣滃垱寤轰簡涓€涓悕涓?SMACTR 鐨勬鏋讹紝璇ユ鏋朵娇宸ョ▼甯堝彲浠ュ鏍?AI 绯荤粺銆傝繘琛岃繖椤瑰伐浣滅殑鍘熷洜鏄负浜嗚В鍐崇洰鍓嶈娑堣垂鑰呭箍娉涗娇鐢ㄤ互渚涗娇鐢ㄧ殑 AI 绯荤粺瀛樺湪鐨勯棶璐ｅ埗宸窛銆傚湪杩欓噷闃呰瀹屾暣鐨勬姤鍛?sup class="mq-185">[27]锛屽湪杩欓噷闃呰瀹屾暣鐨勮鏂?sup class="mq-186">[28]銆?/p>
鍦∟LP Highlights[29]鎾鐨勬柊鍓ч泦涓紝Thomas Wolf 鍜?Victor Sanh 璁ㄨ浜嗘ā鍨嬭捀棣忥紝浠ュ強濡備綍灏嗗叾鐢ㄤ綔鍘嬬缉澶у瀷妯″瀷锛堝 BERT锛変互鐢ㄤ簬鍙墿灞曠殑瀹為檯 NLP 搴旂敤绋嬪簭鐨勫彲琛屾柟娉曘€備粬浠湪浠栦滑鎻愬嚭鐨勭О涓篋istilBERT[30]鐨勬柟娉曚腑瀵规姒傚康杩涜浜嗚繘涓€姝ョ殑璁ㄨ锛屽湪璇ユ柟娉曚腑锛屼粬浠瀯寤鸿緝灏忕殑妯″瀷锛堝熀浜庤緝澶фā鍨嬬殑鐩稿悓浣撶郴缁撴瀯锛夛紝浠ユ牴鎹妯″瀷鐨勮緭鍑烘潵妯′豢杈冨ぇ妯″瀷鐨勮涓恒€傛湰璐ㄤ笂锛岃緝灏忕殑妯″瀷锛堝鐢燂級浼氬皾璇曟牴鎹叾杈撳嚭鍒嗗竷鏉ユ嫙鍚堟暀甯堢殑姒傜巼鍒嗗竷銆?/p>
鏈€杩戯紝鍏充簬璇稿 BERT 鐨勪笂涓嬫枃鍖栨柟娉曟垚鍔熺敤浜庡鐞嗗悇绉嶅鏉傜殑 NLP 浠诲姟鐨勮璁哄緢澶氥€傚湪杩欑瘒鏂囩珷涓紝Kawin Ethayarajh 璇曞浘鍥炵瓟浠ヤ笅闂锛氳濡?BERT锛孍LMo 鍜?GPT-2 涔嬬被鐨勪笂涓嬫枃妯″瀷鍙婂叾涓婁笅鏂囧寲鐨勮瘝琛ㄧず褰㈠紡鏄粈涔?sup class="mq-197">[31]锛熶富棰樺寘鎷澧冩€э紝璇鐗瑰畾鎬х殑搴﹂噺浠ュ強闈欐€佸祵鍏ヤ笌璇鍖栬〃绀轰箣闂寸殑姣旇緝銆?/p>
ML 鐮旂┒浜哄憳 Fran莽oisLagunas 鍐欎簡杩欑瘒寰堟鐨勬枃绔狅紝Is the future of Neural Networks Sparse?[32] 璁ㄨ浜嗕粬瀵瑰湪绁炵粡缃戠粶妯″瀷涓噰鐢ㄧ█鐤忓紶閲忕殑涔愯鎬佸害銆傚笇鏈涢噰鐢ㄦ煇绉嶅舰寮忕殑绋€鐤忔€ф潵鍑忓皬褰撳墠妯″瀷鐨勫ぇ灏忥紝杩欎簺妯″瀷鐢变簬鍏跺ぇ灏忓拰閫熷害鑰屽湪鏌愪簺鏃跺€欏彉寰椾笉鍒囧疄闄呫€傜敱浜庡綋鍓嶆ā鍨嬶紙渚嬪 Transformer锛夌殑搴炲ぇ瑙勬ā锛堥€氬父渚濊禆鏁板崄浜夸釜鍙傛暟锛夛紝鍥犳鍦?ML 涓彲鑳藉€煎緱鎺㈣杩欎竴姒傚康銆備絾鏄紝浠庡紑鍙戜汉鍛樺伐鍏风殑瑙掑害鏉ョ湅锛屽湪 GPU 涓婄殑绁炵粡缃戠粶涓敮鎸佹湁鏁堢█鐤忔€х殑瀹炵幇缁嗚妭灏氫笉娓呮锛岃繖鏄満鍣ㄥ涔犵ぞ鍖烘鍦ㄥ姫鍔涚殑浜嬫儏銆?/p>
濡傛灉浣犳兂瀛︿範濡備綍浠庨浂寮€濮嬭缁冭瑷€妯″瀷[33]锛岃鏌ョ湅 Hugging Face 鐨勮繖浠戒护浜哄嵃璞℃繁鍒讳笖鍏ㄩ潰鐨勬暀绋嬨€備粬浠樉鐒跺埄鐢ㄤ簡鑷繁鐨勫簱 Transformers 鍜?Tokenizers 鏉ヨ缁冩ā鍨嬨€?/p>
Cathal Horan 鍙戣〃浜嗕竴绡囦护浜哄嵃璞℃繁鍒讳笖闈炲父璇︾粏鐨勫崥瀹㈡枃绔狅紝Tokenizers: How machines read[34]锛屽唴瀹规秹鍙婃渶鏂扮殑 NLP 妯″瀷濡備綍浠ュ強浣跨敤鍝绫诲瀷鐨?tokenizer 鏉ュ府鍔╂満鍣ㄥ涔犵畻娉曚粠鏂囨湰淇℃伅涓涔犮€備粬杩樿璁哄苟婵€鍙戜簡涓轰粈涔?tokenizer 鏄縺鍔ㄤ汉蹇冧笖閲嶈鐨勭爺绌舵椿璺冮鍩熺殑鍘熷洜銆傛枃绔犵敋鑷宠繕灞曠ず浜嗗浣曚娇鐢?SentencePiece 鍜?WordPiece 绛夋爣璁板寲鏂规硶鏉ヨ缁冭嚜宸辩殑鏍囪鍖栧伐鍏枫€?img data-ratio="0.7105643994211288" src="/img?url=https://mmbiz.qpic.cn/mmbiz_png/DHibuUfpZvQc59wGKwjDC7oibrKLghA7HeFrXKUN8ibBc6pyKfMzxDdwyFl2YtlgiaBcCcEpOQZH1R145sNObRVzkg/640?wx_fmt=png" data-type="png" data-w="691" _width="100%" class="mq-210" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?> 鐜板湪锛屼綘鍙互鍦ㄧ嚎瀛︿範 2020 MLVU 鏈哄櫒瀛︿範璇剧▼[35]锛屽叾涓寘鎷叏濂楀够鐏墖锛岃棰戝拰鏁欏澶х翰銆傚畠鏃ㄥ湪浣滀负 ML 鐨勫叆闂紝浣嗗畠涔熷寘鍚叾浠栦笌娣卞害瀛︿範鐩稿叧鐨勪富棰橈紝渚嬪 VAE 鍜?GAN銆?/p>
SuzanaIli膰 鍜屼笢浜満鍣ㄥ涔狅紙MLT锛夊湪浣?ML 鏁欒偛姘戜富鍖栨柟闈竴鐩村仛鐫€鎯婁汉鐨勫伐浣溿€備緥濡傦紝璇ュ簱Machine-Learning-Tokyo/Math_resources[36]灞曠ず浜嗗厤璐圭殑鍦ㄧ嚎璧勬簮闆嗗悎锛岀敤浜庡涔?ML 涓娇鐢ㄧ殑鏁板姒傚康鐨勫熀纭€銆?/p>
MIT 鐨勨€滄繁搴﹀涔犲叆闂ㄢ€濊绋?sup class="mq-224">[37]锛?姣忓懆閮戒細鍙戝竷鏂扮殑璁插骇锛屾墍鏈夋柟闈㈠拰瑙嗛锛堝寘鎷紪鐮佸疄楠屽锛夐兘灏嗗彂甯冦€?/p>
Alfredo Canziani 鍙戝竷浜哖yTorch 娣卞害瀛︿範[38]寰瀷璇剧▼鐨勫够鐏墖鍜岀瑪璁版湰锛岃璧勬簮搴撹繕鍖呭惈涓€涓厤濂楃綉绔欙紝鍏朵腑鍖呭惈瀵规湰璇剧▼涓墍鏁欐巿姒傚康鐨勬枃瀛楁弿杩般€?/p>
鐢遍夯鐪佺悊宸ュ闄㈢殑鏁欏憳鍙戝竷鐨勨€淢issing Semester鈥?sup class="mq-232">[39]鏄竴闂ㄥ緢妫掔殑鍦ㄧ嚎璇剧▼锛屽叾涓寘鍚闈炲紑鍙戣儗鏅殑鏁版嵁绉戝瀹惰€岃█鍙兘鏈夌敤鐨勬潗鏂欍€傚畠鍖呮嫭璇稿 Shell 宸ュ叿鍜岃剼鏈互鍙婄増鏈帶鍒朵箣绫荤殑涓婚銆?/p>
CMU 鍙戝竷浜嗏€淎dvanced Deep Learning鈥?sup class="mq-236">[40]璇剧▼鐨勫够鐏墖鍜屾暀瀛﹀ぇ绾诧紝鍏朵腑鍖呮嫭璇稿鑷姩鍥炲綊妯″瀷锛岀敓鎴愭ā鍨嬩互鍙婅嚜鎴戠洃鐫?棰勬祴瀛︿範绛変富棰樸€傝璇剧▼閫傜敤浜?MS 鎴?Ph.D锛?鍏锋湁 ML 楂樼骇鑳屾櫙鐨勫鐢熴€?/p>
BERT-of-Theseus[41]鎻愬嚭浜嗕竴绉嶉€氳繃灏?BERT 妯″瀷鍒掑垎涓哄師濮嬬粍浠舵潵閫愭鏇挎崲鍜屽帇缂?BERT 妯″瀷鐨勬柟娉曘€傞€氳繃閫愭鏇挎崲鍜岃缁冿紝杩樺叿鏈夊皢妯″瀷鐨勫師濮嬬粍浠跺拰鍘嬬缉鐗堟湰缁勫悎鍦ㄤ竴璧风殑浼樺娍銆傛墍鎻愬嚭鐨勬ā鍨嬩紭浜?GLUE 鍩哄噯涓婄殑鍏朵粬鐭ヨ瘑鎻愮偧鏂规硶銆?/p>
杩欏効鎵惧埌涓€浠芥湁瓒ｇ殑璇剧▼锛岀О涓衡€滄満鍣ㄥ涔犲叆闂ㄢ€?sup class="mq-242">[42]锛屾兜鐩栦簡 ML 鍩虹鐭ヨ瘑锛岀洃鐫ｅ洖褰掞紝闅忔満妫灄锛屽弬鏁拌皟鏁翠互鍙婅澶氬叾浠栧熀鏈殑 ML 涓婚銆?/p>
甯岃厞璇?BERT锛圙reekBERT锛?sup class="mq-244">[43]妯″瀷鐜板湪鍙€氳繃 Hugging Face Transformers 搴撲娇鐢ㄣ€?/p>
Jeremy Howard[44]鍙戣〃浜嗕竴绡囪鏂囷紝鎻忚堪浜?fastai 娣卞害瀛︿範搴擄紝璇ュ簱琚箍娉涚敤浜庣爺绌跺苟鏁欐巿鍏舵繁搴﹀涔犲紑鏀捐绋嬶紝 鎺ㄨ崘缁欒嚧鍔涗簬鏋勫缓鍜屾敼杩涙繁搴﹀涔犲拰 ML 搴撶殑杞欢寮€鍙戜汉鍛樸€?/p>
Deeplearning.ai 瀹屾垚浜?TensorFlow 鎵€鏈夊洓涓绋嬬殑鍙戝竷锛歍ensorFlow: Data and Deployment Specialization[45]銆傝涓撲笟鐨勪富瑕佺洰鐨勬槸鏁欒偛寮€鍙戜汉鍛樺浣曞湪涓嶅悓鐨勫満鏅腑鏈夋晥鍦伴儴缃叉ā鍨嬶紝浠ュ強鍦ㄨ缁冩ā鍨嬫椂浠ユ湁瓒ｄ笖鏈夋晥鐨勬柟寮忎娇鐢ㄦ暟鎹€?/p>
Sebastian Raschka 鏈€杩戝彂琛ㄤ簡涓€绡囬涓恒€奝ython 涓殑鏈哄櫒瀛︿範锛氭暟鎹瀛︼紝鏈哄櫒瀛︿範鍜屼汉宸ユ櫤鑳界殑涓昏鍙戝睍鍜屾妧鏈秼鍔?sup class="mq-250">[46]銆嬬殑璁烘枃銆傛湰鏂囨槸瀵规満鍣ㄥ涔犲伐鍏峰墠鏅殑鍏ㄩ潰鍥為【銆傚浜庣悊瑙?ML 宸ョ▼涓娇鐢ㄧ殑鏌愪簺搴撳拰姒傚康鐨勫悇绉嶄紭鐐硅€岃█锛岃繖鏄竴浠芥瀬濂界殑鎶ュ憡銆傛澶栵紝杩樻彁渚涗簡鏈夊叧鍩轰簬 Python 鐨勬満鍣ㄥ涔犲簱鐨勬湭鏉ョ殑淇℃伅銆?/p>
鍥剧伒鑷劧璇█鐢熸垚锛圱-NLG锛? https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/ NVIDIA Megatron-LM: https://github.com/NVIDIA/Megatron-LM Linguistically Informed Neural Dependency Parsing for Typologically Diverse Languages: http://uu.diva-portal.org/smash/record.jsf?pid=diva2:1357373&dswid=7905 渚濊禆鍏崇郴瑙ｆ瀽: http://nlpprogress.com/english/dependency_parsing.html Rapid Adaptation of BERT for Information Extraction on Domain-Specific Business Documents: https://arxiv.org/abs/2002.01861 question understanding benchmark: https://arxiv.org/abs/2001.11770v1 鏈夎叮鐨勫伐浣? https://ai.facebook.com/blog/using-radioactive-data-to-detect-if-a-data-set-was-used-for-training/ 瀹屾暣璁烘枃: https://arxiv.org/pdf/2002.00937.pdf ERNIE-GEN: http://xxx.itp.ac.cn/abs/2001.11314 change.org: https://www.change.org/p/organizers-of-data-science-and-machine-learning-conferences-neurips-icml-aistats-iclr-uai-allow-remote-paper-poster-presentations-at-conferences 鏂板崥瀹? https://yoshuabengio.org/2020/02/10/fusce-risus/ 鎶借薄鎺ㄧ悊璇枡搴擄紙ARC锛? https://www.kaggle.com/c/abstraction-and-reasoning-challenge/overview 鏈哄櫒瀛︿範鍜?NLP 棰嗗煙 2019 骞村嚭鐗堟暟鎹粺璁? https://www.marekrei.com/blog/ml-and-nlp-publications-in-2019/ Growing Neural Cellular Automata: https://distill.pub/2020/growing-ca/ 浜や簰寮?Transformer 娉ㄦ剰鍙鍖? https://github.com/SIDN-IAP/attnvis SketchTransfer: https://arxiv.org/pdf/1912.11570.pdf Python NLP灞曟湜: https://speakerdeck.com/inesmontani/the-future-of-nlp-in-python-keynote-pycon-colombia-2020 DeepSpeed 涓?ZeRO: https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/ PyTorch3D: https://ai.facebook.com/blog/-introducing-pytorch3d-an-open-source-library-for-3d-deep-learning/ Hydra: https://hydra.cc/ CausalNex: https://causalnex.readthedocs.io/en/latest/01_introduction/01_introduction.html TyDi QA: https://ai.googleblog.com/2020/02/tydi-qa-multilingual-question-answering.html 闂瓟搴? https://github.com/huggingface/node-question-answering Forte: https://github.com/asyml/forte 璇嗗埆鏂囨湰淇℃伅涓殑涓昏鍋忚: https://podcasts.apple.com/us/podcast/will-ai-help-identify-bias-or-perpetuate-it-with-diyi-yang/id1435564422?i=1000464141922 DeepMind: https://deepmind.com/research/publications/Artificial-Intelligence-Values-and-Alignment 瀹屾暣鐨勬姤鍛? https://venturebeat.com/2020/01/30/google-researchers-release-audit-framework-to-close-ai-accountability-gap/ 瀹屾暣鐨勮鏂? https://dl.acm.org/doi/abs/10.1145/3351095.3372873 NLP Highlights: https://soundcloud.com/nlp-highlights/104-model-distillation-with-victor-sanh-and-thomas-wolf DistilBERT: https://arxiv.org/abs/1910.01108 璇稿 BERT锛孍LMo 鍜?GPT-2 涔嬬被鐨勪笂涓嬫枃妯″瀷鍙婂叾涓婁笅鏂囧寲鐨勮瘝琛ㄧず褰㈠紡鏄粈涔? https://kawine.github.io/blog/nlp/2020/02/03/contextual.html Is the future of Neural Networks Sparse?: https://medium.com/huggingface/is-the-future-of-neural-networks-sparse-an-introduction-1-n-d03923ecbd70 浠庨浂寮€濮嬭缁冭瑷€妯″瀷: https://huggingface.co/blog/how-to-train Tokenizers: How machines read: https://blog.floydhub.com/tokenization-nlp/ MLVU 鏈哄櫒瀛︿範璇剧▼: https://mlvu.github.io/ Machine-Learning-Tokyo/Math_resources: https://github.com/Machine-Learning-Tokyo/Math_resources MIT 鐨勨€滄繁搴﹀涔犲叆闂ㄢ€濊绋? http://introtodeeplearning.com/ PyTorch 娣卞害瀛︿範: https://atcold.github.io/pytorch-Deep-Learning-Minicourse/ 鈥淢issing Semester鈥? https://missing.csail.mit.edu/ 鈥淎dvanced Deep Learning鈥? https://andrejristeski.github.io/10707-S20/syllabus.html BERT-of-Theseus: http://xxx.itp.ac.cn/abs/2002.02925 鈥滄満鍣ㄥ涔犲叆闂ㄢ€? https://compstat-lmu.github.io/lecture_i2ml/index.html 甯岃厞璇?BERT锛圙reekBERT锛? https://huggingface.co/nlpaueb/bert-base-greek-uncased-v1 Jeremy Howard: https://arxiv.org/abs/2002.04688 TensorFlow: Data and Deployment Specialization: https://www.coursera.org/specializations/tensorflow-data-and-deployment Python 涓殑鏈哄櫒瀛︿範锛氭暟鎹瀛︼紝鏈哄櫒瀛︿範鍜屼汉宸ユ櫤鑳界殑涓昏鍙戝睍鍜屾妧鏈秼鍔? https://arxiv.org/abs/2002.048031.6 REALM: Retrieval-Augmented Language Model Pre-Training
1.7 ERNIE-GEN
2銆丆reativity and Society 馃帹
2.1 鍏佽鍦ㄧ瀛︿細璁笂杩涜杩滅▼璁烘枃鍜屾捣鎶ュ睍绀?/span>
2.2 Abstraction and Reasoning Challenge
2.3 ML and NLP Publications in 2019
2.4 Growing Neural Cellular Automata
Morphogenesis鏄竴涓嚜缁勭粐杩囩▼锛岄€氳繃璇ヨ繃绋嬶紝铦捐瀳绛夋煇浜涚敓鐗╁彲浠ュ啀鐢熸垨淇韬綋鎹熶激銆傝杩囩▼瀵逛簬鎵板姩鏄瞾妫掔殑锛屽苟涓旀湰璐ㄤ笂鏄嚜閫傚簲鐨勩€傚彈杩欑鐢熺墿瀛︾幇璞＄殑鍚彂锛岄渶瑕佹洿濂藉湴浜嗚В杩欎竴杩囩▼锛岀爺绌朵汉鍛樺彂琛ㄤ簡涓€绡囬涓?Growing Neural Cellular Automata"[14]鐨勮鏂囷紝璇ヨ鏂囬噰鐢ㄤ簡鍙尯鍒嗙殑褰㈡€佸彂鐢熸ā鍨嬶紝鏃ㄥ湪澶嶅埗鑷垜淇绯荤粺鐨勮涓哄拰鐗规€э紝骞跺笇鏈涜兘澶熷埗閫犲嚭鍏锋湁涓庣敓鐗╃敓鍛界浉鍚岀殑鍧氬浐鎬у拰鍙鎬х殑鑷垜淇 machines銆傛澶栵紝杩欏皢鏈夊姪浜庢洿濂藉湴浜嗚В鍐嶇敓杩囩▼鏈韩锛?璁稿搴旂敤娉ㄥ叆鍐嶇敓鍖诲浠ュ強绀句細鍜岀敓鐗╃郴缁熺殑寤烘ā绛夐兘灏嗗彈鐩娿€?img data-ratio="0.5357142857142857" src="/img?url=https://mmbiz.qpic.cn/mmbiz_png/DHibuUfpZvQc59wGKwjDC7oibrKLghA7HeRVzMk9L0It6uWodV7rJQTaPM1fwIYP2hJfu1ueROIDmPPnoshBibC7Q/640?wx_fmt=png" data-type="png" data-w="532" _width="100%" class="mq-123" alt="寤鸿鏀惰棌锛佹渶鍏ㄨ嚜鐒惰瑷€澶勭悊鏃朵簨绠€鎶?>2.5 Transformer attention 鍙鍖?/span>
2.6 SketchTransfer
2.7 The future of NLP in Python
3銆乀ools and Datasets 鈿欙笍
3.1 DeepSpeed + ZeRO
3.2 涓€涓敤浜庤繘琛屽揩閫熸湁鏁堢殑 3D 娣卞害瀛︿範鐮旂┒鐨勫簱
3.3 绠＄悊浣犵殑 ML 椤圭洰閰嶇疆
3.4 璐濆彾鏂綉缁滃洜鏋滄帹鐞嗗伐鍏峰寘
3.5 TyDi 闂瓟锛氬璇█闂瓟鍩哄噯
3.6 Question Answering for Node.js
3.7 鐏垫椿鑰屽己澶х殑NLP宸ュ叿鍖?/span>
4銆丒thics in AI 馃毃
4.1 璇嗗埆鏂囨湰涓殑 subjective bias
4.2 Artificial Intelligence, Values and Alignment
4.3 鍏充簬瀹℃牳 AI 绯荤粺
5銆丄rticles and Blog posts 鉁嶏笍
5.1 鐢ㄤ簬 NLP 绯荤粺鐨勬ā鍨嬭捀棣?/span>
5.2 鍗曡瘝鐨勪笂涓嬫枃琛ㄧず
5.3 绁炵粡缃戠粶涓殑绋€鐤忔€?/span>
5.4 璁粌浣犺嚜宸辩殑璇█妯″瀷
5.5 鍒嗚瘝鍣?Tokenizer
6銆丒ducation 馃帗
6.1 闃垮鏂壒涓硅嚜鐢卞ぇ瀛︽満鍣ㄥ涔犺绋?/span>
6.2 鏈哄櫒瀛︿範鏁板璧勬簮
6.3 娣卞害瀛︿範鍏ラ棬
6.4 Pytorch 娣卞害瀛︿範
6.5 Missing Semester of Your CS
6.6 娣卞害瀛︿範杩涢樁
7銆丯oteworthy Mentions 猸愶笍
鏈枃鍙傝€冭祫鏂?/span>
銆怭ython涓枃绀惧尯涓撳睘浼樻儬鐮併€?/span>
鈻?/span> 鐐瑰嚮闃呰鍘熸枃锛屽嵆浜吘璁簯浜у搧1鎶樹紭鎯犺捣
以上是关于的主要内容，如果未能解决你的问题，请参考以下文章