Skip to content

Commit

Permalink
Limit sysinfo crate FDs usage. (Azure#5055)
Browse files Browse the repository at this point in the history
**Context:**
We use [Sysinfo](https://docs.rs/sysinfo/0.18.0/sysinfo/) crate to get host metrics and processes info. By default it keeps `/proc/...` files open for perf optimizations. Unfortunately, there could be many processes/threads in the host system, so `sysinfo` can consume a lot of FDs.

**Solution:**
After looking into details of `sysinfo`, I found out that we can limit the number of FDs it can use.

Also, maybe we don't need to refresh everything if we don't use that?
  • Loading branch information
vadim-kovalyov authored May 28, 2021
1 parent d0c1bf8 commit bc56061
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions edgelet/edgelet-docker/src/runtime.rs
Original file line number Diff line number Diff line change
Expand Up @@ -336,8 +336,9 @@ impl MakeModuleRuntime for DockerModuleRuntime {
})
.join(notary_registries)
.map(move |(client, (notary_registries, _))| {
let mut system_resources = System::new_all();
system_resources.refresh_all();
// to avoid excessive FD usage, we will not allow sysinfo to keep files open.
sysinfo::set_open_files_limit(0);
let system_resources = System::new_all();
info!("Successfully initialized module runtime");
let notary_lock = tokio::sync::lock::Lock::new(BTreeMap::new());
DockerModuleRuntime {
Expand Down Expand Up @@ -842,7 +843,6 @@ impl ModuleRuntime for DockerModuleRuntime {
.as_ref()
.lock()
.expect("Could not acquire system resources lock");
system_resources.refresh_all();

let current_time = SystemTime::now()
.duration_since(UNIX_EPOCH)
Expand All @@ -851,18 +851,20 @@ impl ModuleRuntime for DockerModuleRuntime {
let start_time = process::id()
.try_into()
.map(|id| {
system_resources.refresh_process(id);
system_resources
.get_process(id)
.map(|p| p.start_time())
.unwrap_or_default()
})
.unwrap_or_default();

system_resources.refresh_system();
let used_cpu = system_resources.get_global_processor_info().get_cpu_usage();

let total_memory = system_resources.get_total_memory() * 1000;
let used_memory = system_resources.get_used_memory() * 1000;

system_resources.refresh_disks();
let disks = system_resources
.get_disks()
.iter()
Expand Down

0 comments on commit bc56061

Please sign in to comment.