"

28 IT On Board

 

Man facing an open laptop in a computer server room.
James Cordray (Marine Computer Specialist, IODP USIO/TAMU) checks the servers in the IT office. (Credit: William Crawford, IODP/TAMU) [Photo ID: exp346_141]

Going out to sea is a feat in itself, but then being tasked with maintaining the technology and internet access while working against the waves is another.

Technology at Sea

In early times of sea expeditions, explorers were completely cut off from the onshore world, save for letters that would make their way to/from ships (see Communication Methods Ship-To-Shore). Over time, limited internet access evolved into today’s seamless connectivity, where smartphones and laptops can operate simultaneously onboard a vessel.  This was a notable difference to sailing scientists that returned after years since their first expedition to find out they no longer had to announce when they were going to go on a Zoom call on JOIDES Resolution.

Marine Computer Specialists (MCS) were in charge of the IODP computer operations, database systems, and technology on the ship. They worked in tandem with the Applications Developers on the ship and the onshore IT teams back at Texas A&M University to ensure operations ran smoothly. Each expedition was different and they came onto the ship knowing what projects to complete to maintain the ship’s technology systems in compliance with the university’s operating protocols.

MCS’s had to create the ship’s network and database as its own ecosystem that could operate within the ship at sea. There were no options for internet routers to connect to utility poles out at sea. The ecosystem consisted of both the internet service to send and receive data from outside the ship, and the intranet service that allowed for shipboard communications and data sharing. The internet was satellite dependent which meant that if the connection to the satellite was lost, due to storms or extensive cloud cover, the intranet would still offer the ability for everyone on the ship to continue working with minimal interruption. However, you would not be able to use external communication tools like email or Zoom calls to anyone on shore.

Since the operating manager of JOIDES Resolution was Sea-1 (previously called Siem Offshore), and IODP served as the client for the ship, there were policies in place where Sea- 1 would have their own internet and intranet networks, servers, and computers for their crew, separate from those used for the IODP technicians and scientists. The technicians and scientists were granted access to certain shared drives that provided a centralized location for that team to upload and share data, reports, photos etc. The IODP shared drives were only accessible through the IODP provided computers and network access. The sailing scientists were given log in credentials to use through their own laptops to access the servers and internet upon their arrival to the ship. The same limitations were placed with the Sea-1 shared drives and computers. Should there have been a Sea-1 crew member that needed access to a IODP shared drive or vice versa, they would have to go through a similar process of being granted access with the MCS’s.

man standing in the computer server room pointing to one of the servers.
Have you ever wondered how we store an expedition’s worth of data? Here, Steve Thomas (Marine Computer Specialist, IODP JRSO) shows us the data center. (Credit: Sandra Herrmann, IODP JRSO) [Photo ID: exp395E_123]

 

While on the ship the scientists were consistently doing data entry, and reporting on weekly operations through reports. As you could imagine, having multiple people access the same file location to edit could lead to files being deleted accidentally. Fortunately, it became practice early on for the MCS’s to routinely back up the servers and shared drives every 6 hours. Not only were the share drives backed up, but the instrument hosts with critical science data were backed up regularly as well. This useful fact helped many deleted files to be recovered in a timely fashion. For example, if you deleted the file on Tuesday night, there was a backup of the shared drive earlier that day. Which means that the MCS’s could return an older version of the file to the team or person in question. Once each expedition was completed, a common practice was to allow everyone an opportunity to download what expedition science data they needed. After all science party members and crew departed the ship, all the shared drive directories were wiped clean.

A crowded room filled with people sitting on every side of a table on their laptops.
Science meeting in the conference room. (Credit: Froukje van der Zwan & IODP) [Photo ID: exp368_030]

Onboarding Visiting Scientists

A women presenting to a room full of people about the upcoming expedition.
Leah LeVay (Expedition Project Manager/Staff Scientist, IODP JRSO) at the first meeting of Expedition 395 in the conference room (after waiting 3 years!). (Credit: Saran Lee & IODP) [Photo ID: 004_exp395_C0025.jpg]

First day of getting on the ship all visiting scientists, since most were new to the JR each expedition, were oriented on internet vs intranet services, permissible internet use, and device connection.

Some of the most recent requirements included:

  • Do not stream or watch youtube videos
  • Do not run any music streaming services and to make sure to download all your music before coming on the ship or leaving port
  • The JR business network only allowed for one device to have internet access per science party participant – meaning you get to use only your phone or laptop to have access to the internet that than had access to the servers.
  • In year 2024, to get around the one device per JR network, Starlink was enlisted to provide participants with less restrictive access to wireless internet, the difference being the Starlink could not be used to access the intranet.
  • Before IODP Expedition 401 in 2023, one device was allowed for internet use while on expedition.  Then for and after Expedition 401, with the addition of the Starlink satellite internet service, it allowed for you to have access to the internet on multiple devices.
  • Storms will interfere with internet access

 

The above slides are taken from the IT orientation powerpoint presentation given to IODP Expedition 402 scientists (Credit: IODP JRSO)

Even if you were the most technologically savvy person, you might still have encountered a bit of confusion because of how different the IT ecosystem was from what you found at your home institution. Usually there were a few people that had to do a few extra steps before successfully joining the IT ecosystem. Usually due to a security feature on the device, an old software, you didn’t do all the steps in the right order, or you got given the wrong password. The MCSs have encountered their fair share of issues that, for the most part, they knew how to troubleshoot during the onboarding process. However, there is always something that becomes a new lesson learned.

The first step for any and all members beign granted access to the internet was that each device that was going to be connected to the shipboard wireless network to access IODP servers and shared drives, would need to install a Network Access Control (NAC)  agent software. After connecting to the shipboard wireless network, the NAC agent would prompt for installation. It was like a digital fingerprint, allowing the ship’s IT systems to track connected devices and give them proper network access.

Given time and experience, MCS’s anticipated issues like:

  • If people were bringing laptops that were company owned, and they do not have the administrative authority to make any changes, which made  it much harder to install the NAC Agent or printer drivers needed for onboard printing. It soon became obvious to warn people before to ask their own IT department to be given temporary admin access during the expedition.
  • Texas A&M University set minimum requirements for operating systems (OS) that could be given access to the shipboard network which could access any expedition science data. There were always 2 or 3 people that could not participate with that device because its OS was not compatible with those requirements. Although it was not their responsibility, the IT team would try to help in any way possible to procure a copy of the OS they could use to update their laptop or help with OS upgrading before allowing network access. This was implemented to help prevent cybersecurity risks because if the OS was not receiving security updates any further, they were not allowed access to the shipboard business network. Great examples of this are Windows 7 and macOS Mojave.
  • Had to start warning oncoming scientists to either deactivate any softwares that require two factor authentication or find other options to use while on the ship because you can’t get the text message or push notification. The MCS’s had to create a more open wifi network that they could temporarily give someone access to receive the internet based push notification (if it was text message then you would have to wait until back onshore).
  • It is likely that iMessage or WhatsApp on the laptops will eventually go out of sync because iCloud access is denied, so one had to choose their method of communication to shore wisely.

 

As you worked to log on to the internet, and intranet, one constant message from the MCSs was to make sure that all auto updates, and auto save functions are completely turned off on your devices for the time of the expedition. The reason being that if the auto save or auto update functions were left on, the internet consumption of the ship would go through the roof and  subsequently slow down all the systems. Meaning, you could not depend on using iCloud Storage, Adobe Creative Cloud, or Google Drive because they all use a form of auto updating and cloud storage functions. If at any point your digital footprint grew exorbitantly even for less than an hour, the MCS could shut down access to the internet on your device without getting out of their chair. Then they will work with you to determine what software on your device was using the internet to upload to a cloud.

 

All ecosystems require a balance and the MCSs are there meticulously ensuring the ship’s balance is maintained.

 

SciOD Spotlight – Marine Computer Specialists

Read below how the sailing MCS’s in 2024 had to pivot when the temporary Microsoft outage affected technology at sea. The text is from a JOIDES Resolution blog post written by IODP Expedition 403 Outreach Officer, Tim Lyons, titled Tech Turmoil Finds Its Way Into the Arctic (July 22, 2024).

 

Two of the marine computer specialists standing next to each other in their office posing for the photograph
IODP Expedition 403 Marine Compute Specialists, Nick Logan and Kerry Mullins. (Credit: Tim Lyons, IODP Expedition 403 Outreach Officer.)

The group WhatsApp chat is typically reserved for memes and misplaced laundry, but two days ago, the expedition’s project manager wrote a long message to explain why things had come to a sudden halt. “All lab computers are down,” began the message that arrived sometime around 7am. A quick response by Isuri Kapuge, a member of the sedimentology team, explained that the issue was worldwide.

Working in the Arctic Circle at times feels like being on another planet. This feeling is only amplified by the constant daylight and often late arrival of news. Yesterday morning, however, a global software malfunction found its way onto this planet and shut down most of the lab-based computers. The dreadful and familiar “blue screen of death” appeared at the start of the day across most working computers. More than just an inconvenience, the bug brought coring operations to a halt because the computer necessary for measuring the samples that determine the probability of encountering problematic gases was one of the many blue screens.

“It’s a double edged sword, how much we rely on technology. Without it, we couldn’t do anything that we do out here,” Explains Kerry Mullins, the marine computer specialist onboard. What began as a single computer down, quickly became most of the ship. The problem only worsened when Kerry learned the computer responsible for managing headspace samples was down. Headspace samples measure the gas contents of the cores coming in, and the ratio of heavy to light gases helps predict how close a dangerous oil or gas reservoir could be to the drill site. Without having this data, it is like driving with a blindfold on, and for the safety of the expedition, coring operations had to be put on pause until the computer was up and running again. Kerry goes on to say, “You never want to wake up your shift mate, and you do everything you can to avoid it, but after finding a quick patch to fix the problem, the decision was made to wake up Nick to help implement it.”

“When you see a few missed calls from your boss in the middle of the night, you know things are bad.” Nick Logan was woken up before the start of his shift to help get the systems back up. He jumped into action, bringing the servers back up online. “Things could have been a lot worse, in so many ways we got lucky that we didn’t have a perfect storm come together, if so we could have been sitting stranded like many others still are” explains Nick. The two recount the story of how things unfolded with humility often making a point to explain how much luck was on their side in the moment. These expeditions can take decades to plan, and while at sea, every moment counts. Whatever luck was on the side of this expedition, without question, the speed at which operations were able to resume was the result of quick action on Nick and Kerry’s part. What could have lasted days, only lasted a few hours.

Quick thinking and quick action got everything up and running as quickly, luck only assisted this fact. Kerry explained to me that it was a single line of unchecked code in the kernel that was released that caused the entire meltdown. Much of the world is still affected by this line of code: flights are still not rescheduled, businesses are still not running, and University servers are still down.

It is scary to think of the speed at which the problem arrived at our research ship so far north. Another reminder that although the Arctic feels like another planet, it very much is not, and the issues that are faced elsewhere can find their way up here faster than we expect. Perhaps this is an illustration of why the climate research on this expedition is so important; the world is smaller than we think, and change can come faster than we want.

 

License

Icon for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Scientific Ocean Drilling: Exploration and Discovery through Time Copyright © 2024 by Laura Guertin; Elizabeth Doyle; and Tessa Peixoto is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.