Linux Systems Engineer 2
Employer: Pacific Northwest National Laboratory · Location: Richland, Washington · Posting: PNL-12066 · Type: Full-Time · Pay: $89,300-$131,200/yr
Duties
Overview At PNNL, our core capabilities are divided among major departments that we refer to as Directorates within the Lab, focused on a specific area of scientific research or other function, with its own leadership team and dedicated budget. Our Science & Technology directorates include National Security, Integrated Discovery Sciences, and Energy and Environment. In addition, we have an Environmental Molecular Sciences Laboratory, a Department of Energy, Office of Science user facility housed on the PNNL campus. The Integrated Discovery Sciences Directorate (IDSD) leads fundamental research across biology, chemistry, Earth and environmental sciences, materials science, advanced computing, artificial intelligence, quantum information science, mathematics, autonomy, and DOE national user facilities. Our vision is to accelerate scientific discovery by observing, understanding, simulating, predicting, and controlling dynamic biotic-abiotic processes and interactions within complex biological, chemical, material, and Earth systems. We intentionally connect disciplines to advance fundamental discovery science, develop transformative scientific capabilities, and address the nation's most important challenges in energy, environmental resilience, biotechnology, advanced manufacturing, health, and national security. The Environmental Molecular Sciences Division is comprised of 18 interdisciplinary research teams focused on deciphering molecular-level interactions driving biological and environmental processes across temporal and spatial scales. Through computational analysis and modeling, these findings contribute to predictive understanding of how systems respond to environmental perturbations thus enabling solutions to the nation’s energy, environmental, and human health challenges. The division also manages the Environmental Molecular Sciences Laboratory, a Department of Energy, Office of Science user facility housed on the PNNL campus that accelerates the research of scientists around the world by providing access to world-class expertise, instrumentation, and computational resources. EMSL is a U.S. Department of Energy (DOE) Office of Science user facility that provides innovative and breakthrough experimental and computational science supporting DOE Office of Biological and Environmental Research (BER) programs by offering access to more than 75 state of the art instrumental and high performance computing capabilities. Responsibilities We are seeking a Linux Systems Engineer to operate and expand the Computing and Data Operations (CDO) Linux infrastructure and high performance computing clusters that support EMSL's scientific instruments. This role covers hands on hardware work, Linux systems management, configuration automation, and Slurm based cluster operations across CDO's compute clusters (Tahoma and Boreal) and storage systems, including Ceph, VastData, BeeGFS, Lustre, and the Aurora/HPSS archive. The ideal candidate brings strong experience with on premises or hybrid Linux environments, modern automation tooling, and the ability to work collaboratively within a small, mission focused engineering team. Install, configure, maintain, and troubleshoot Linux servers (RHEL/Rocky and derivatives) across CDO's HPC clusters and EMSL support infrastructure, including bare-metal provisioning. Administer HPC cluster operations on Tahoma and Boreal, including Slurm scheduler configuration, node provisioning with Warewulf, InfiniBand networking, and hardware diagnostics. Develop and maintain automation using Ansible, including playbooks, roles, and configuration pipelines. Support and understanding of large-scale storage systems (Ceph, VastData, BeeGFS, Lustre, and the Aurora/HPSS archive) underlying HPC compute and EMSL instrument data. Work with DevOps tooling and workflows to ensure reliable software and infrastructure deployments (GitLab CI/CD, Git ‑ based workflows). Participate in on ‑ call or rotational support for mission ‑ critical systems. Monitor compute, storage, and network health using tools such as Prometheus, Grafana, Nagios, and ELK. Assist with containerized workloads (Docker, Kubernetes) where applicable to HPC operational workflows. Perform hands-on hardware work on-site — racking, cabling, component replacement, and diagnostics — as a regular part of data center operations. Document processes, procedures, and checklists to ensure operational consistency across environments. Collaborate closely with development, research, and infrastructure teams to improve cluster performance, reliability, and automation. This position requires onsite work a minimum of three days per week. Regular hands on hardware work, including racking, cabling, component replacement, and diagnostics in the data center, is a core and ongoing part of this role. No security clearance is required. Qualifications Minimum Qualifications: BS/BA and 2 years of relevant experience -OR- MS/MA -OR- PhD Preferred Qualifications: Degree in Computer Science, Computer Information Systems, or a related field. Strong Linux administration background including installation, patching, tuning, and troubleshooting. Hands on automation experience with Ansible (YAML, roles, playbooks) or other system configuration tool. Experience with virtualization technologies or cloud platforms (VirtualBox, Proxmox, AWS, Azure), or hybrid deployments. Experience with Databases such as MySQL, MariaDB, Postgres, or MongoDB. Scripting skills (bash, Python). Comfort and physical ability to perform hands-on server hardware work — diagnostics, firmware updates, racking, and cabling — on-site in a data center environment. Familiarity with GitLab CI or Github Actions and Git-centric workflows. Basic knowledge of containerization (Docker/Kubernetes) used in DevOps and ML/HPC contexts. Experience deploying or supporting HPC systems or large Linux installations. Experience administering an HPC job scheduler (Slurm preferred). Experience with bare-metal provisioning tools such as xCAT or Warewulf. Experience using linux system packaging tools to deploy, remove, and package software. Experience with large-scale storage systems such as Ceph, VastData, Lustre, BeeGFS, or HPSS. Experience building dashboards, alerting, or automation on top of monitoring stacks such as Prometheus, Grafana, or ELK. Experience collaborating with AI on scripting, software development, and/or system administration work. Hazardous Working Conditions/Environment Not applicable. Testing Designated Position This is not a Testing Designated Position (TDP). About PNNL Pacific Northwest National Laboratory (PNNL) is a world-class research institution powered by a highly educated, diverse workforce committed to the values of Integrity, Creativity, Collaboration, Impact, and Courage. Every year, scores of dynamic, driven people come to PNNL to work with renowned researchers on meaningful science, innovations and outcomes for the U.S. Department of Energy and other sponsors; here is your chance to be one of them! At PNNL, you will find an exciting research environment and excellent benefits including health insurance, and flexible work schedules. PNNL is located in eastern Washington State—the dry side of Washington known for its stellar outdoor recreation and affordable cost of living. The Lab’s campus is only a 45-minute flight (or ~3 hour drive) from Seattle or Portland, and is serviced by the convenient PSC airport, connected to 8 major hubs. Commitment to Excellence and Equal Employment Opportunity Our laboratory is committed to fostering a work environment where all individuals are treated with fairness and respect while solving critical challenges in fundamental sciences, national security, and energy resiliency. We are an Equal Employment Opportunity employer. Pacific Northwest National Laboratory (PNNL) is an Equal Opportunity Employer. PNNL considers all applicants for employment without regard to race, religion, color, sex, national origin, age, disability, genetic information (including family medical history), protected veteran status, and any other status or characteristic protected by federal, state, and/or local laws. We are committed to providing reasonable accommodations for individuals with disabilities and disabled veterans in our job application procedures and in employment. If you need assistance or an accommodation due to a disability, contact us at careers@pnnl.gov . Drug Free Workplace PNNL is committed to a drug-free workplace supported by Workplace Substance Abuse Program (WSAP) and complies with federal laws prohibiting the possession and use of illegal drugs. If you are offered employment at PNNL, you must pass a drug test prior to commencing employment. PNNL complies with federal law regarding illegal drug use. Under federal law, marijuana remains an illegal drug. If you test positive for any illegal controlled substance, including marijuana, your offer of employment will be withdrawn. Security, Credentialing, and Eligibility Requirements As a national laboratory, PNNL is responsible for adhering to the Homeland Security Presidential Directive 12 (HSPD-12) and Department of Energy (DOE) Order 473.1A, which require new employees to obtain and maintain a HSPD-12 Personal Identify Verification (PIV) Credential. To obtain this credential, new employees must successfully complete the applicable tier of federal background investigation post hire and receive a favorable federal adjudication. The tier of federal background investigation will be determined by job duties and national security or public trust responsibilities associated with the job. All tiers of investigation include a declaration of illegal drug activities, including use, supply, possession, or manufacture within the last 1 to 7 years (depending on the applicable tier of investigation). Illegal drug activities include marijuana and cannabis derivatives, which are still considered illegal under federal law, regardless of state laws. For foreign national candidates: If you have not resided in the U.S. for three consecutive years, you are not eligible for the PIV credential and instead will need to obtain a favorable Local Site Specific Only (LSSO) Federal risk determination to maintain employment. Once you meet the three-year residency requirement thereafter, you will be required to obtain a PIV credential to maintain employment. The tier of federal background investigation required to obtain the PIV credential will be determined by job duties at the time you become eligible for the PIV credential. Mandatory Requirements Please be aware that the Department of Energy (DOE) prohibits DOE employees and contractors from having any affiliation with the foreign government of a country DOE has identified as a “country of risk” without explicit approval by DOE and Battelle. If you are offered a position at PNNL and currently have any affiliation with the government of one of these countries, you will be required to disclose this information and recuse yourself of that affiliation or receive approval from DOE and Battelle prior to your first day of employment. Rockstar Rewards Employees and their families are offered medical insurance, dental insurance, vision insurance, robust telehealth care options, several mental health benefits, free wellness coaching, health savings account, flexible spending accounts, basic life insurance, disability insurance*, employee assistance program, business travel insurance, tuition assistance, relocation, backup childcare, legal benefits, supplemental parental bonding leave, surrogacy and adoption assistance, and fertility support. Employees are automatically enrolled in our company-funded pension plan* and may enroll in our 401 (k) savings plan with company match*. Employees may accrue up to 120 vacation hours per year and may receive ten paid holidays per year. * Research Associates excluded. **All benefits are dependent upon eligibility. Click Here For Ro
Qualifications
Minimum Qualifications: BS/BA and 2 years of relevant experience -OR- MS/MA -OR- PhD Preferred Qualifications: Degree in Computer Science, Computer Information Systems, or a related field. Strong Linux administration background including installation, patching, tuning, and troubleshooting. Hands on automation experience with Ansible (YAML, roles, playbooks) or other system configuration tool. Experience with virtualization technologies or cloud platforms (VirtualBox, Proxmox, AWS, Azure), or hybrid deployments. Experience with Databases such as MySQL, MariaDB, Postgres, or MongoDB. Scripting skills (bash, Python). Comfort and physical ability to perform hands-on server hardware work — diagnostics, firmware updates, racking, and cabling — on-site in a data center environment. Familiarity with GitLab CI or Github Actions and Git-centric workflows. Basic knowledge of containerization (Docker/Kubernetes) used in DevOps and ML/HPC contexts. Experience deploying or supporting HPC systems or large Linux installations. Experience administering an HPC job scheduler (Slurm preferred). Experience with bare-metal provisioning tools such as xCAT or Warewulf. Experience using linux system packaging tools to deploy, remove, and package software. Experience with large-scale storage systems such as Ceph, VastData, Lustre, BeeGFS, or HPSS. Experience building dashboards, alerting, or automation on top of monitoring stacks such as Prometheus, Grafana, or ELK. Experience collaborating with AI on scripting, software development, and/or system administration work.
Compensation
Minimum Salary USD $89,300.00/Yr. Maximum Salary USD $131,200.00/Yr.