linux fopen个数限制的问题（文件描述符限制）

Posted 2021-02-02 airduce

问题出现的情况大概是这样的：我有一个多线程程序，每个线程是要fopen一个文件。当打开到第900+个的时候，程序突然异常退出，原因是fopen得到的为空，及不能再打开文件了。这是我查看了一下进程最大打开文件描述符数：ulimit -Sn ,值是1024。ulimit -Hn,值是4096。我就怀疑是这个1024限制了线程的打开文件数。接下来我就参照以下文档改这个值。

ile Descriptor Requirements (Linux Systems)

To ensure good server performance, the total number of client connections, database files, and log files must not exceed the maximum file descriptor limit on the operating system (ulimit -n). By default, the directory server allows an unlimited number of connections but is restricted by the file descriptor limit on the operating system. Linux systems limit the number of file descriptors that any one process may open to 1024 per process. (This condition is not a problem on Solaris machines, x86, x64, or SPARC).

After the directory server has exceeded the file descriptor limit of 1024 per process, any new process and worker threads will be blocked. For example, if the directory server attempts to open a Oracle^® Berkeley JE database file when the operating system has exceeded the file descriptor limit, the directory server will no longer be able to open a connection that can lead to a corrupted database exception. Likewise, if you have a directory server that exceeds the file descriptor limit set by the operating system, the directory server can become unresponsive as the LDAP connection handler consumes all of the CPU\'s processing in attempting to open a new connection.

To fix this condition, set the maximum file descriptor limit per process on Linux machines.

 

To Increase the File Descriptor Limit (Linux)

1. Display the current hard limit of your machine.

   The hard limit is the maximum server limit that can be set without tuning the kernel parameters in proc file system.

   $ ulimit -aH
   core file size (blocks)       unlimited
   data seg size (kbytes)        unlimited
   file size (blocks)            unlimited
   max locked memory (kbytes)    unlimited
   max memory size (kbytes)      unlimited
   open files                    1024
   pipe size (512 bytes)         8
   stack size (kbytes)           unlimited
   cpu time (seconds)            unlimited
   max user processes            4094
   virtual memory (kbytes)       unlimited

2. Edit the /etc/security/limits.conf and add the lines:

   *     soft   nofile  1000000
   *     hard   nofile  1000000

3. Edit the /etc/pam.d/login by adding the line:

   session required /lib/security/pam_limits.so

4. Use the system file limit to increase the file descriptor limit to 65535.

   The system file limit is set in /proc/sys/fs/file-max .

   echo 1000000> /proc/sys/fs/file-max

5. Use the ulimit command to set the file descriptor limit to the hard limit specified in/etc/security/limits.conf.

   ulimit -n 1000000

6. Restart your system.

　　　　reboot

再次查看unlimit -Sn ,值为1000000，unlimit -Hn，值为1000000.

再次跑我的程序的时候，顺利突破1000的大关。这会儿已经跑到2000+了也没出问题。

参考https://www.cnblogs.com/qq78292959/archive/2012/03/07/2383337.html。

参考https://blog.csdn.net/chinaclock/article/details/48346117?utm_source=blogkpcl8。

感谢这两位博主的无私奉献！