{"id":1181,"date":"2024-04-24T07:31:31","date_gmt":"2024-04-24T07:31:31","guid":{"rendered":"https:\/\/www.nicekj.com\/?p=1181"},"modified":"2024-04-24T07:32:44","modified_gmt":"2024-04-24T07:32:44","slug":"jiyualiyunmianfeisuanlizijianllmleigptdamoxing","status":"publish","type":"post","link":"https:\/\/www.nicekj.com\/jiyualiyunmianfeisuanlizijianllmleigptdamoxing.html","title":{"rendered":"\u57fa\u4e8e\u963f\u91cc\u4e91\u514d\u8d39\u7b97\u529b\u81ea\u5efaLLM(\u7c7bGPT)\u5927\u6a21\u578b"},"content":{"rendered":"<p>\u6211\u6b63\u5728\u53c2\u52a0\u300c\u6398\u91d1\u00b7\u542f\u822a\u8ba1\u5212\u300d<\/p>\n<blockquote>\n<p>\u76ee\u524d\u5f00\u6e90\u7684 <code>LLM<\/code> \u5927\u6a21\u578b\uff0c\u8981\u60f3\u6bd4\u8f83\u6d41\u7545\u5730\u8fdb\u884c\u90e8\u7f72\u4f53\u9a8c\uff0c\u79bb\u4e0d\u5f00 <code>GPU<\/code> \u7b97\u529b\u3002\u672c\u6587\u4ecb\u7ecd\u57fa\u4e8e\u963f\u91cc\u4e91\u7684\u673a\u5668\u5b66\u4e60\u5e73\u53f0 <code>PAI<\/code> \u6765\u514d\u8d39\u4f53\u9a8c\u4e00\u4e9b\u5f00\u6e90\u5927\u6a21\u578b\u3002<\/p>\n<\/blockquote>\n<h2 data-id=\"heading-0\">\u83b7\u53d6\u7b97\u529b<\/h2>\n<p>\u76f4\u63a5\u6253\u5f00\u963f\u91cc\u4e91\u514d\u8d39\u6d3b\u52a8\u9875\u9009\u62e9<a href=\"https:\/\/link.juejin.cn?target=https%3A%2F%2Ffree.aliyun.com%2F%3Fproduct%3D9602825\" target=\"_blank\" title=\"https:\/\/free.aliyun.com\/?product=9602825\" ref=\"nofollow noopener noreferrer\" rel=\"noopener\">\u673a\u5668\u5b66\u4e60\u5e73\u53f0PAI<\/a>\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/9009a5b2ebc6eadf30667a83322ab13f.png\" alt=\"pai\" \/><\/figure>\n\n\u76f4\u63a5\u70b9\u51fb\u7acb\u5373\u8bd5\u7528\uff0c\u6ce8\u610f\uff0c<strong>\u65b0\u8001\u7528\u6237<\/strong>\u90fd\u53ef\u4ee5\uff0c\u6240\u4ee5\u653e\u5fc3\u7533\u8bf7\u3002\u4e00\u822c PAI \u529f\u80fd\u6d89\u53ca\u5230\u5b58\u50a8\uff0c\u6240\u4ee5\u5efa\u8bae\u540c\u65f6\u4e5f\u7533\u8bf7 <code>NAS<\/code> \u4f53\u9a8c\u529f\u80fd\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/a56e0db533bcce3d9c5703f60d2cf0c3.png\" alt=\"ali_nas\" \/><\/figure>\n<\/p>\n<p>\u5982\u679c\u4f60\u9700\u8981\u5f15\u5165 <code>NAS<\/code> \u6301\u4e45\u5316\u6570\u636e\u5c31\u5230<a href=\"https:\/\/link.juejin.cn?target=https%3A%2F%2Fnasnext.console.aliyun.com%2Foverview%3Fspm%3Da2c4g.2261408.0.0.35993a02KpRU0s\" target=\"_blank\" title=\"https:\/\/nasnext.console.aliyun.com\/overview?spm=a2c4g.2261408.0.0.35993a02KpRU0s\" ref=\"nofollow noopener noreferrer\" rel=\"noopener\">\u63a7\u5236\u53f0<\/a>\u65b0\u5efa\u5373\u53ef\uff0c\u8fd9\u91cc\u4e0d\u518d\u8d58\u8ff0\u3002<\/p>\n<h3 data-id=\"heading-1\">\u65b0\u5efa\u7a7a\u95f4<\/h3>\n<p>\u7533\u8bf7\u7ed3\u675f\u540e\u76f4\u63a5\u65b0\u5efa\u4e00\u4e2a<a href=\"https:\/\/link.juejin.cn?target=https%3A%2F%2Fpai.console.aliyun.com%2F%3FregionId%3Dcn-shenzhen%26scm%3D20140722.M_960469._.V_1%26spm%3D5176.28008736%23%2Fworkspace%2Fcreate\" target=\"_blank\" title=\"https:\/\/pai.console.aliyun.com\/?regionId=cn-shenzhen&amp;scm=20140722.M_960469._.V_1&amp;spm=5176.28008736#\/workspace\/create\" ref=\"nofollow noopener noreferrer\" rel=\"noopener\">\u5de5\u4f5c\u7a7a\u95f4<\/a>\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/441004152fd05d61c190db099bf69d92.png\" alt=\"pai_ws\" \/><\/figure>\n<\/p>\n<p>\u8d44\u6e90\u6d88\u8017\u5728<a href=\"https:\/\/link.juejin.cn?target=https%3A%2F%2Fusercenter2.aliyun.com%2Fri%2Fsummary%3Fspm%3Da2c4g.2261127.0.0.1928229272eccf%26commodityCode%3D\" target=\"_blank\" title=\"https:\/\/usercenter2.aliyun.com\/ri\/summary?spm=a2c4g.2261127.0.0.1928229272eccf&amp;commodityCode=\" ref=\"nofollow noopener noreferrer\" rel=\"noopener\">\u8d44\u6e90\u5b9e\u4f8b\u7ba1\u7406<\/a>\u67e5\u770b\u3002<\/p>\n<h3 data-id=\"heading-2\">\u521b\u5efaDSW<\/h3>\n<p>\u521b\u5efa\u597d\u7a7a\u95f4\uff0c\u76f4\u63a5\u521b\u5efa\u4ea4\u4e92\u5f0f\u5efa\u6a21\uff08DWS\uff09\u5b9e\u4f8b\uff0c\u8fd9\u91cc\u6211\u4eec\u9700\u8981\u6ce8\u610f\u7684\u662f\u53ea\u80fd\u9009\u62e9\u53ef\u4ee5\u62b5\u6263\u7b97\u529b\u7684 <code>GPU<\/code>\uff1a<\/p>\n<ul>\n<li>ecs.gn7i-c8g1.2xlarge: A10<\/li>\n<li>ecs.gn6v-c8g1.2xlarge: V100<\/li>\n<\/ul>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/a6e2ecfe8bb42cb9626b462849be0cd6.png\" alt=\"pai_dsw\" \/><\/figure>\n<\/p>\n<p>\u955c\u50cf\u9009\u62e9 <code>pytorch:1.12-gpu-py39-cu113-ubuntu20.04<\/code> \u5373\u53ef\uff0c\u521b\u5efa\u5b8c\u6bd5\u5728**\u4ea4\u4e92\u5f0f\u5efa\u6a21\uff08DSW\uff09**\u9009\u4e2d\u521b\u5efa\u7684\u76ee\u6807\u70b9\u51fb\u6253\u5f00\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/120ed10520a138aa95b0b65513b68cde.png\" alt=\"dsw_llm\" \/><\/figure>\n<\/p>\n<p>\u5177\u4f53\u64cd\u4f5c\u9875\u9762\u5982\u4e0b\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/ace8d7ae6fd745ede6c692f0bb6d9ef0.png\" alt=\"dsw_jupyter\" \/><\/figure>\n<\/p>\n<p>\u5c31\u662f\u57fa\u4e8e <code>**Jupyter Lab**<\/code> \u4fee\u6539\u7684\u754c\u9762\uff0c\u5f88\u597d\u4e0a\u624b\uff0c\u63a5\u4e0b\u6765\u5c31\u8ba9\u6211\u4eec\u7528\u8fd9\u4e2a\u514d\u8d39\u7684\u7b97\u529b\u6765\u4f53\u9a8c\u4e00\u4e0b\u5404\u7c7b\u5f00\u6e90 <code>LLM<\/code> \u5927\u6a21\u578b\u5427\ud83e\udd73\u3002<\/p>\n<p>\u5173\u4e8e\u4f7f\u7528\u963f\u91cc\u4e91\u4e5f\u5f88\u8d34\u5fc3\u5730\u51fa\u4e86<a href=\"https:\/\/link.juejin.cn?target=https%3A%2F%2Fhelp.aliyun.com%2Fdocument_detail%2F2261126.html%3Fspm%3D5176.28008736.J_6443120770.d960469_1.42b03e4dtkYl8d%26pipCode%3Dlearn%26goodsId%3D960469%26scm%3D20140722.M_960469._.V_1\" target=\"_blank\" title=\"https:\/\/help.aliyun.com\/document_detail\/2261126.html?spm=5176.28008736.J_6443120770.d960469_1.42b03e4dtkYl8d&amp;pipCode=learn&amp;goodsId=960469&amp;scm=20140722.M_960469._.V_1\" ref=\"nofollow noopener noreferrer\" rel=\"noopener\">\u6559\u7a0b<\/a>\u7ed9\u6211\u4eec\u4e0a\u624b\u4f7f\u7528\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/86d943155d4c711a6e83a2769284b1af.png\" alt=\"PAI_HELP\" \/><\/figure>\n<\/p>\n<h2 data-id=\"heading-3\">\u4e0a\u624b LLM<\/h2>\n<h3 data-id=\"heading-4\">\u51c6\u5907\u5de5\u4f5c<\/h3>\n<p>\u63a5\u4e0b\u6765\u5c06\u4ee5\u5f00\u6e90\u9886\u57df\u6bd4\u8f83\u51fa\u540d\u7684\u51e0\u4e2a <code>LLM<\/code> \u4e3a\u4f8b\uff0c\u8dd1\u8d77\u6765\u4f53\u9a8c\u4e00\u4e0b\uff0c\u5f00\u59cb\u524d\u505a\u597d\u4e00\u4e9b\u51c6\u5907\u5de5\u4f5c\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">\u5b89\u88c5 git-lfs<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\">curl -s https:\/\/packagecloud.io\/install\/repositories\/github\/git-lfs\/script.deb.sh | sudo bash<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"3\">apt-get install git-lfs<\/span>\n<\/code><\/pre>\n<h3 data-id=\"heading-5\">ChatGLM-6B<\/h3>\n<p>\u4e0b\u8f7d\u9879\u76ee\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">https:\/\/github.com\/THUDM\/ChatGLM-6B.git<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">\u56fd\u5185\u52a0\u901f<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"3\">git clone https:\/\/ghproxy.com\/https:\/\/github.com\/THUDM\/ChatGLM-6B.git<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"4\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">\u5b89\u88c5\u4f9d\u8d56<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"5\">pip install -r requirements.txt<\/span>\n<\/code><\/pre>\n<p>\u52a0\u8f7d\u6a21\u578b\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">mkdir -p  \/mnt\/workspace\/chatglm-6b<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\">git clone https:\/\/huggingface.co\/THUDM\/chatglm-6b \/mnt\/workspace\/chatglm-6b<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"3\">cd  \/mnt\/workspace\/chatglm-6b<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"4\">git lfs install<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"5\">git lfs pull<\/span>\n<\/code><\/pre>\n<p>\u82e5\u901f\u5ea6\u6162\uff0c<a href=\"https:\/\/link.juejin.cn?target=https%3A%2F%2Fgithub.com%2FTHUDM%2FChatGLM-6B\" target=\"_blank\" title=\"https:\/\/github.com\/THUDM\/ChatGLM-6B\" ref=\"nofollow noopener noreferrer\" rel=\"noopener\">\u5b98\u65b9<\/a>\u63d0\u4f9b\u7684\u624b\u52a8\u4e0b\u8f7d\u6a21\u578b\u6587\u4ef6\u65b9\u6848\u4e5f\u53ef\u53c2\u8003\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/11aaf4b188e58a5e7eb1cb4e0ac1d2e6.png\" alt=\"chatglm-model\" \/><\/figure>\n<\/p>\n<p>\u8fd0\u884c\u9879\u76ee\uff0c\u57fa\u4e8e <code>transformers<\/code> \u5feb\u901f\u4f7f\u7528\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/a153cdf64ceace72c3c2c71a2a4b78e9.png\" alt=\"chatglm-transformers\" \/><\/figure>\n<\/p>\n<p><code>ChatGLM-6B<\/code> \u63d0\u4f9b\u4e86 <code>cli&amp;web&amp;api<\/code> \u4e09\u79cd\u4f7f\u7528\u65b9\u5f0f\uff0c\u4f7f\u7528\u524d\u8bf7\u5c06\u8fd9\u4e09\u4e2a\u6587\u4ef6\u5168\u90e8\u4fee\u6539\u4e0b\u6a21\u578b\u76ee\u5f55\uff1a<\/p>\n<ul>\n<li>web_demo.py\uff1a\u8bbe\u7f6e <code>share=True<\/code> \u53ef\u4ee5\u5206\u4eab\u51fa\u53bb<\/li>\n<li>cli_demo.py<\/li>\n<li>api.py<\/li>\n<\/ul>\n<p>\u6bd4\u5982\u6211\u6f14\u793a\u73af\u5883\u6a21\u578b\u76ee\u5f55\u4e0b\u8f7d\u4f4d\u7f6e\u662f <code>\/mnt\/workspace\/chatglm-6b<\/code>\uff0c\u6539\u52a8\u540e\u4ee3\u7801\u5982\u4e0b\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">tokenizer = AutoTokenizer.from_pretrained(\"\/mnt\/workspace\/chatglm-6b\", trust_remote_code=True)<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\">model = AutoModel.from_pretrained(\"\/mnt\/workspace\/chatglm-6b\", trust_remote_code=True).half().cuda()<\/span>\n<\/code><\/pre>\n<p>\u63a5\u4e0b\u6765\u542f\u52a8\u8fd0\u884c\u5bf9\u5e94\u811a\u672c\u5373\u53ef\u4f53\u9a8c\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">\u4ee5\u7ec8\u7aef\u4e3a\u4f8b<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\">python cli_demo.py<\/span>\n<\/code><\/pre>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/a85820298e84b12ca6e505188dc63e9d.png\" alt=\"chatglm-cli-demo\" \/><\/figure>\n<\/p>\n<h3 data-id=\"heading-6\">MOSS<\/h3>\n<p>\u51c6\u5907\u597d\u9879\u76ee\u548c\u6a21\u578b\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">git clone https:\/\/github.com\/OpenLMLab\/MOSS.git<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">\u5b89\u88c5\u4f9d\u8d56<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"3\">pip install -r requirements.txt<\/span>\n<\/code><\/pre>\n<p>\u7531\u4e8e\u786c\u4ef6\u95ee\u9898\uff0c\u6211\u4eec\u4f7f\u7528<code>4bit<\/code>\u91cf\u5316\u7248\u672c\u7684<code>moss-moon-003-sft<\/code>\u6a21\u578b(\u9ed8\u8ba4)\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">mkdir -p  \/mnt\/workspace\/moss-moon-003-sft-int4<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\">git clone https:\/\/huggingface.co\/fnlp\/moss-moon-003-sft-int4 \/mnt\/workspace\/moss-moon-003-sft-int4<\/span>\n<\/code><\/pre>\n<p>\u7136\u540e\u5c06<code>moss_cli_demo.py<\/code>\u7684 <code>32-34<\/code> \u884c\u4ee3\u7801\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">model_path = args.model_name<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\">if not os.path.exists(args.model_name):<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"3\">    model_path = snapshot_download(args.model_name)<\/span>\n<\/code><\/pre>\n<p>\u6539\u4e3a\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">model_path = args.model_name<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\"><span class=\"hljs-keyword\">if<\/span> not os.path.exists(args.model_name):<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"3\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">    model_path = snapshot_download(args.model_name)<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"4\">model_path = \"\/mnt\/workspace\/moss-moon-003-sft-int4\"<\/span>\n<\/code><\/pre>\n<p>\u7136\u540e\u6267\u884c\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">python moss_cli_demo.py<\/span>\n<\/code><\/pre>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/9ef706619406fd361f15eab19ec3f513.png\" alt=\"moss-cli\" \/><\/figure>\n<\/p>\n<p>\u5982\u679c\u60f3\u4f7f\u7528 web \u4ea4\u4e92\u7248\u672c\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">python moss_web_demo_gradio.py<\/span>\n<\/code><\/pre>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/79c9fa40d9c5fc82d0a1fe7c0891c680.png\" alt=\"moss-web\" \/><\/figure>\n<\/p>\n<h3 data-id=\"heading-7\">baichuan-7B<\/h3>\n<p>\u51c6\u5907\u597d\u9879\u76ee\u548c\u6a21\u578b\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">git clone https:\/\/github.com\/baichuan-inc\/baichuan-7B.git<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">\u56fd\u5185\u52a0\u901f<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"3\">git clone https:\/\/ghproxy.com\/https:\/\/github.com\/baichuan-inc\/baichuan-7B.git<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"4\"><span class=\"hljs-meta prompt_\"># <\/span><span class=\"bash\">\u5b89\u88c5\u4f9d\u8d56<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"5\">pip install -r requirements.txt<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"6\"><span class=\"hljs-meta prompt_\"><\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"7\"># <span class=\"bash\">\u4e0b\u8f7d\u6a21\u578b<\/span><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"8\">mkdir -p  \/mnt\/workspace\/baichuan-7b<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"9\">git clone https:\/\/huggingface.co\/baichuan-inc\/baichuan-7B \/mnt\/workspace\/baichuan-7b<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"10\">cd \/mnt\/workspace\/baichuan-7b<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"11\">git lfs install<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"12\">git lfs pull<\/span>\n<\/code><\/pre>\n<p>\u8fd0\u884c\u9879\u76ee\uff0c\u57fa\u4e8e <code>transformers<\/code> \u5feb\u901f\u4f7f\u7528\uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">pip install accelerate<\/span>\n<\/code><\/pre>\n<p>\u53ea\u80fd\u8bf4\uff0c\u52c9\u5f3a\u80fd\u8dd1\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/514a10ee20ddd934794040fac0bbaa12.png\" alt=\"baichuan-transformers\" \/><\/figure>\n<\/p>\n<p>\u60f3\u8981\u4f53\u9a8c\u5bf9\u8bdd\u80fd\u529b\uff1f\u6709\u5927\u4f6c\u5df2\u7ecf\u5fae\u8c03\u4e86\u5bf9\u5e94\u7248\u672c\uff0c\u5982 <a href=\"https:\/\/link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fhiyouga%2Fbaichuan-7b-sft\" target=\"_blank\" title=\"https:\/\/huggingface.co\/hiyouga\/baichuan-7b-sft\" ref=\"nofollow noopener noreferrer\" rel=\"noopener\">baichuan-7b-sft<\/a> \uff1a<\/p>\n<pre><\/div><div class=\"code-block-extension-headerRight\"><span class=\"code-block-extension-lang\">shell<\/span><div class=\"code-block-extension-copyCodeBtn\">\u590d\u5236\u4ee3\u7801<\/div><\/div><\/div><code class=\"hljs language-shell code-block-extension-codeShowNum\" lang=\"shell\"><span class=\"code-block-extension-codeLine\" data-line-num=\"1\">mkdir -p  \/mnt\/workspace\/baichuan-7b-sft<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"2\">mkdir -p \/mnt\/workspace\/baichuan-7b-sft-offload-dir<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"3\">git clone https:\/\/huggingface.co\/hiyouga\/baichuan-7b-sft \/mnt\/workspace\/baichuan-7b-sft<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"4\">cd  \/mnt\/workspace\/baichuan-7b-sft<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"5\">git lfs install<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"6\">git lfs pull<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"7\"><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"8\">git clone https:\/\/github.com\/hiyouga\/LLaMA-Efficient-Tuning<\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"9\"><\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"10\">python src\/cli_demo.py <\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"11\">    --model_name_or_path \/mnt\/workspace\/baichuan-7b <\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"12\">    --checkpoint_dir \/mnt\/workspace\/baichuan-7b-sft <\/span>\n<span class=\"code-block-extension-codeLine\" data-line-num=\"13\">    --prompt_template ziya<\/span>\n<\/code><\/pre>\n<p>\u6267\u884c\u7ed3\u679c\uff1a<\/p>\n<p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.nicekj.com\/wp-content\/uploads\/replace\/7e7b280e94539fab0b9ae9155c01634b.png\" alt=\"baichuan-sft-cli\" \/><\/figure>\n<\/p>\n<h2 data-id=\"heading-8\">\u8bf4\u660e<\/h2>\n<p>\u611f\u8c22\u963f\u91cc\u4e91\u7684\u514d\u8d39\u8ba1\u5212\uff0c\u6574\u4f53\u4f53\u9a8c\u4e0b\u6765\u611f\u89c9\u8fd8\u662f\u4e0d\u9519\u7684\uff0c\u5176\u5b9e\u5c31\u76f8\u5f53\u4e8e\u53ef\u9009\u7684 <code>A10&amp;V100 GPU<\/code> \u8ba9\u4f60\u4f53\u9a8c\u4f7f\u7528\uff0c\u5927\u5bb6\u53ef\u4ee5\u53d1\u6325\u60f3\u8c61\u529b\u6765\u4f7f\u7528\uff0c\u6709\u4ec0\u4e48\u95ee\u9898\u6b22\u8fce\u6c9f\u901a\u4ea4\u6d41\u3002<\/p>\n<p>\u611f\u8c22\u4f60\u9605\u8bfb\u5230\u8fd9\u91cc\uff0c\u5982\u679c\u6b64\u6587\u5bf9\u4f60\u6709\u5e2e\u52a9\uff0c\u6b22\u8fce\u8f6c\u53d1\u70b9\u8d5e\u3002\ud83d\udc6c\ud83c\udffb \u670b\u53cb\uff0c\u90fd\u770b\u5230\u8fd9\u4e86\uff0c\u786e\u5b9a\u4e0d\u5173\u6ce8\u4e00\u4e0b\u4e48 \ud83d\udc47<\/p>","protected":false},"excerpt":{"rendered":"<p>\u6211\u6b63\u5728\u53c2\u52a0\u300c\u6398\u91d1\u00b7\u542f\u822a\u8ba1\u5212\u300d \u83b7\u53d6\u7b97\u529b \u76f4\u63a5\u6253\u5f00\u963f\u91cc\u4e91\u514d\u8d39\u6d3b\u52a8\u9875\u9009\u62e9\u673a\u5668\u5b66\u4e60\u5e73\u53f0PAI\uff1a \u76f4\u63a5\u70b9\u51fb\u7acb\u5373\u8bd5\u7528\uff0c\u6ce8\u610f\uff0c\u65b0\u8001\u7528\u6237\u90fd\u53ef\u4ee5\uff0c\u6240\u4ee5\u653e\u5fc3\u7533\u8bf7\u3002\u4e00\u822c PAI \u529f\u80fd\u6d89\u53ca\u5230\u5b58\u50a8\uff0c\u6240\u4ee5\u5efa\u8bae\u540c\u65f6\u4e5f\u7533\u8bf7 NAS<\/p>\n","protected":false},"author":1,"featured_media":2212,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"rank_math_title":"\u514d\u8d39\u4f53\u9a8cLLM\u5927\u6a21\u578b\uff0c\u963f\u91cc\u4e91PAI\u7b97\u529b\u52a9\u60a8\u542f\u822a - \u5948\u65af\u79d1\u6280\u793e\u533a","rank_math_description":"\u672c\u6587\u4ecb\u7ecd\u5982\u4f55\u5229\u7528\u963f\u91cc\u4e91\u673a\u5668\u5b66\u4e60\u5e73\u53f0PAI\u7684\u514d\u8d39\u7b97\u529b\u4f53\u9a8cChatGLM\u3001MOSS\u3001baichuanB\u7b49\u5f00\u6e90\u5927\u6a21\u578b\uff0c\u5e76\u63d0\u4f9b\u8be6\u7ec6\u6b65\u9aa4\u548c\u4ee3\u7801\u793a\u4f8b\uff0c\u52a9\u60a8\u8f7b\u677e\u4e0a\u624b\u4f53\u9a8cLLM\u7684\u5f3a\u5927\u529f\u80fd\u3002","rank_math_focus_keyword":"\u963f\u91cc\u4e91, LLM, \u5927\u6a21\u578b, \u514d\u8d39\u4f53\u9a8c, GPU\u7b97\u529b","views":"49","footnotes":""},"categories":[3],"tags":[127,128,129,136,126],"collection":[],"class_list":["post-1181","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-fenlei2","tag-ai","tag-128","tag-129","tag-136","tag-gpt"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/posts\/1181","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/comments?post=1181"}],"version-history":[{"count":0,"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/posts\/1181\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/media\/2212"}],"wp:attachment":[{"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/media?parent=1181"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/categories?post=1181"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/tags?post=1181"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.nicekj.com\/nicekj2024\/wp\/v2\/collection?post=1181"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}