使用 nfs-client 包上传/下载文件在 n 次迭代后不起作用

Posted 2023-02-16

【中文标题】使用 nfs-client 包上传/下载文件在 n 次迭代后不起作用

【英文标题】：File upload/download using nfs-client package not working after n iteration

【发布时间】：2021-03-15 16:33:56

【问题描述】：

我正在使用nfs-client包实现NFS协议连接linux服务器进行文件上传/下载。基本上目的是NFS协议测试。

这是我的示例代码库

import com.emc.ecs.nfsclient.nfs.io.Nfs3File;
import com.emc.ecs.nfsclient.nfs.io.NfsFileInputStream;
import com.emc.ecs.nfsclient.nfs.io.NfsFileOutputStream;
import com.emc.ecs.nfsclient.nfs.nfs3.Nfs3;
import com.emc.ecs.nfsclient.rpc.CredentialUnix;
int thread_count = ctx.getThreadNum();
String lrandom_name = service_name + "_" + replica_id + "_" + thread_count;

public static void uploadFileToNfs() 
        String localDir = "F:\\look\\"+lrandom_name;
        InputStream inputStream = null;
        OutputStream outputStream = null;
        try 
            //Create a local file object
            File localFile = new File(localDir);
            //Get the file name of the local file, this name is used to create a file with the same name in the specified directory on the remote Nfs server
            String localFileName = localFile.getName();
            Nfs3 nfs3 = new Nfs3(NFS_IP, NFS_DIR, new CredentialUnix(0, 0, null), 3);
            //Create Nfs file object on remote server
            Nfs3File NfsFile = new Nfs3File(nfs3, "/" + localFileName);
            //Open a file input stream
            inputStream = new BufferedInputStream(new FileInputStream(localFile));
            //Open a remote Nfs file output stream and copy the file to the destination
            outputStream = new BufferedOutputStream(new NfsFileOutputStream(NfsFile));

            //Buffer memory
            byte[] buffer = new byte[1024];
            while ((inputStream.read(buffer)) != -1) 
                outputStream.write(buffer);
            
            System.out.println("File upload complete!");
         catch (Exception ex) 
            ex.printStackTrace();
         finally 
            try 
                if (outputStream != null) 
                    outputStream.close();
                
                if (inputStream != null) 
                    inputStream.close();
                
             catch (IOException ex) 
                ex.printStackTrace();
            
        
    

我正在从 Jmeter 运行脚本，但经过 n 次迭代（线程方式）后，我遇到了错误。从 NFS 下载文件的类似脚本以线程方式并行运行。

com.emc.ecs.nfsclient.nfs.NfsException: rpc error, server: 192.168.0.101, RPC error: RPC call is ACCEPTED, but the status is not success, acceptStat=4
        at com.emc.ecs.nfsclient.rpc.RpcWrapper.handleRpcException(RpcWrapper.java:311)
        at com.emc.ecs.nfsclient.rpc.RpcWrapper.callRpcWrapped(RpcWrapper.java:159)
        at com.emc.ecs.nfsclient.nfs.nfs3.Nfs3.wrapped_sendWrite(Nfs3.java:756)
        at com.emc.ecs.nfsclient.nfs.nfs3.Nfs3.wrapped_sendWrite(Nfs3.java:90)
        at com.emc.ecs.nfsclient.nfs.io.NfsFileBase.write(NfsFileBase.java:866)
        at com.emc.ecs.nfsclient.nfs.io.NfsFileOutputStream.writeBufferToFile(NfsFileOutputStream.java:296)
        at com.emc.ecs.nfsclient.nfs.io.NfsFileOutputStream.write(NfsFileOutputStream.java:262)
        at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
        at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:127)
        at java.base/java.io.FilterOutputStream.write(FilterOutputStream.java:108)
        at java_io_FilterOutputStream$write.call(Unknown Source)
        at com.emc.ecs.nfsclient.nfs.io.Script1.run(Script1.groovy:77)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:317)
        at org.codehaus.groovy.jsr223.GroovyCompiledScript.eval(GroovyCompiledScript.java:71)
        at java.scripting/javax.script.CompiledScript.eval(CompiledScript.java:89)
        at org.apache.jmeter.util.JSR223TestElement.processFileOrScript(JSR223TestElement.java:217)
        at org.apache.jmeter.protocol.java.sampler.JSR223Sampler.sample(JSR223Sampler.java:72)
        at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:635)
        at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558)
        at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489)
        at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256)
        at java.base/java.lang.Thread.run(Thread.java:834)

更新 基于反馈而不是所有线程（用户）的一个静态文件，现在我有每个线程（用户）的唯一文件来上传/下载

更新

尝试了以下另一种用例，从一个 NFS 共享文件夹随机选择文件到另一个，同样的问题来了。仅怀疑 NFS 配置有问题

package com.emc.ecs.nfsclient.nfs.io;

import org.junit.Test;
import com.emc.ecs.nfsclient.nfs.NfsSetAttributes;
import com.emc.ecs.nfsclient.nfs.io.Nfs3File;
import com.emc.ecs.nfsclient.nfs.io.NfsFileInputStream;
import com.emc.ecs.nfsclient.nfs.io.NfsFileOutputStream;
import com.emc.ecs.nfsclient.nfs.nfs3.Nfs3;
import com.emc.ecs.nfsclient.rpc.CredentialUnix;
import java.util.Random;


import java.io.IOException;

import org.junit.Test;
String nfs_folder = vars.get("nfs_folder");
String files_folder = vars.get("files_folder");
String NFS_IP = vars.get("server_ip");
String NFS_DIR = "/"+nfs_folder+"/"+files_folder;
String NFS_DIR_ROOT = "/"+nfs_folder;

int thread_count = ctx.getThreadNum();
String service_name = vars.get("Service_Name");
String replica_id = vars.get("Replica_ID");
String lrandom_name = service_name + "_" + replica_id + "_" + thread_count;
int transfer_size = Integer.parseInt(vars.get("transfer_size"))


InputStream inputStream = null;
OutputStream outputStream = null;
try 
  Nfs3 nfs3 = new Nfs3(NFS_IP, NFS_DIR, new CredentialUnix(0, 0, null), 3);
  Nfs3 nfs4 = new Nfs3(NFS_IP, NFS_DIR_ROOT, new CredentialUnix(0, 0, null), 3);
  
  nfsFiles = new Nfs3File(nfs3,"/");
  List<String> children = nfsFiles.list();
  Random randomGenerator = new Random();
  int fileIndex = randomGenerator.nextInt(children.size());
   OUT.println(fileIndex)
  String NfsFileDir = "/"+children.get(fileIndex);
  OUT.println(children.size())
  OUT.println(children.get(fileIndex))
  //Create Nfs file object on remote server
  Nfs3File nfsFile = new Nfs3File(nfs3, NfsFileDir);
  Nfs3File NfsFile = new Nfs3File(nfs4, "/" + lrandom_name);
  //String localFileName = localDir + lrandom_name;
  //Create a local file object
  //File localFile = new File(localFileName);
  //Open a file input stream
  inputStream = new BufferedInputStream(new NfsFileInputStream(nfsFile));
  //Open a remote Nfs file output stream and copy the file to the destination
  outputStream = new BufferedOutputStream(new NfsFileOutputStream(NfsFile));

  //Buffer memory
  byte[] buffer = new byte[transfer_size];

  while (inputStream.read(buffer) != -1) 
      outputStream.write(buffer);
  
  System.out.println("File copy complete!");
 catch (IOException ex) 
  ex.printStackTrace();
 finally 
  try 
      if (outputStream != null) 
          outputStream.close();
      
      if (inputStream != null) 
          inputStream.close();
      
   catch (Exception e) 
      e.printStackTrace();
  

【问题讨论】：

看起来像来自 NFS 的 Upload 和 Download 并行执行是这里的问题。如果我一次执行单个操作，则看不到任何问题

【参考方案1】：

我不认为您的用例是有效的，因为将相同的文件上传到相同的目的地并不是真正的 NFS 用户会做的事情，所以也许值得考虑使用即Directory Listing Config plugin，这样每个虚拟用户都会有自己的自己的文件上传/下载

查看NFS RPC RFC你得到的状态是：

GARBAGE_ARGS  = 4, /* procedure can't decode params         */

请仔细检查您的 JMeter 实例是否有足够的空间来运行 JVM 设置、CPU、RAM 等，因为看起来您的 NFS 服务器无法正确解析请求，因此 JMeter 可能会发送一些垃圾.确保关注JMeter Best Practices

与第 2 点基本相同，但对于 NFS 服务器端，可能会出现过载，因此无法正确解析传入的有效请求、检查 NFS 日志和操作系统日志中是否存在任何可疑条目。

【讨论】：

感谢您给我指点和方向。我将根据您的输入更改用例。

每个线程（用户）上传/下载的唯一文件没有解决问题。查看其他指针