PMACS NewsletterSummer 2024

During the early morning hours of July 19^th, Crowdstrike quietly pushed out a minor update to millions of devices worldwide, an activity done routinely multiple times per day. Within minutes, it was clear something had gone wrong. Computers all over the globe suddenly went offline, getting caught in a reboot cycle and showing the dreaded blue screen.

From very early Friday morning, PMACS started its response to address the situation. The Windows systems administrators worked diligently all-day Friday to restore service, bringing servers back online and ensuring file shares were available to the PSOM community. By early afternoon, over 130 Windows servers were restored and fully online.

Over the next three business days, more than 25 PMACS staff worked onsite each day to restore service across the PSOM community, including volunteers from other teams. Each device was in need of direct manual contact in order to resolve the issue. To facilitate faster resolution times, PMACS staffed a walk-up service in the BRB Lobby on Monday and Tuesday, and also conducted multiple building walkthroughs each day to restore as many devices as quickly as possible. Overall, PMACS restored over 1,500 devices in the first three days, which expedited PSOM’s recovery.

I would like to take a moment to extend my gratitude to the dedicated staff who support PSOM every day. Everyone answered the call and stepped up to help PSOM in a challenging moment. I would also extend my appreciation to you, the PSOM community, for your understanding during a moment of crisis. Your patience allowed us to focus on restoring service to as many individuals as possible as quickly as possible.

- Chris Dymek, PSOM Entity Information Officer

Tim Sears was promoted to the position of Project Manager for the Client Services Group (CSG) within Penn Medicine Academic Computing Services (PMACS) at Penn Medicine. In his new role, Tim is responsible for leading key Information Services (IS) operational processes, including but not limited to quality and process improvement initiatives and specific projects in support of PSOM’s needs, such as the annual BGS and UME device rollouts. Tim comes to this position after serving in a Local Support Provider (LSP) role on the CSG team since May 2022. In this role, Tim provided technical knowledge and customer service to PSOM users for break-fix issues. Additionally, during this time, Tim provided support to multiple CSG-wide projects. Before joining Penn Medicine, Tim served as a service desk analyst at financial and educational institutions.

Kaitlyn Phelan was promoted to the position of Application Manager for the Laboratory Information Systems Management (LIMS) team at Penn Medicine. In her new role, Kaitlyn will lead the LIMS team, and oversee the enterprise Research Lab System (LabVantage LIMS), our homegrown biobanking solutions Pumpkin and SPS, and the Xybion pre-clinical management system. With a solid foundation in both science and technology and a BS in Biology from Drexel, Kaitlyn has been instrumental in significant projects such as the LabVantage rollout in the Abramson Cancer Center Clinical Research Unit. Her expertise in LabVantage LIMS and deep relationships within the research community position her to drive strategic initiatives forward.

Kaitlyn's leadership is further recognized through her contributions at key conferences and her role in earning the prestigious 2023 CIO 100 award for Penn Medicine. As she takes on this new challenge, Kaitlyn continues her professional development as an MBA candidate, underscoring her dedication to innovation and excellence in the strategic technology and innovation management sectors.

Penn Medicine transitioned to Microsoft’s latest email security system, Exchange Online Protection (EOP). EOP is a cloud-based filtering service that leverages machine learning and artificial intelligence to protect our organization against spam, malware, phishing, and other email threats. This will change how you report spam and phishing emails and access your quarantine. Read below to learn how your email experience will change.

To report messages as Junk, Phishing, and Not Junk:
The email protection system leverages a New Report Button. Depending on how you access email, you will now see one of these buttons to report mail:

Selecting the Report button offers three options:

Report a Junk Message: Click the Report button and select the Junk option for unwanted or spam messages.

Report a Phishing Message: Click the Report button and select the Phishing option for mail that you believe is Phishing (Malicious). Note that if you successfully identify a cybersecurity phishing simulation, you will receive an email notification advising that you have identified a phishing simulation.

Report an Incorrectly Classified Junk Message: Click the Report button and select the Not Junk option for a message sent to your Junk Folder that you believe is not spam.

Read more on Reporting Emails.

Where will Junk and Quarantined messages get delivered?

• From Outlook, review your Junk Mail folder often, looking for spam messages you may consider legitimate that were incorrectly classified as junk.
• Sign into https://security.microsoft.com/quarantine to access your email Quarantine.

Read More on Quarantine Messages.

Are there other changes? For a complete list of changes to your email experience, please review our Exchange Online Protection documentation.

Penn Medicine now offers a secure and cost-effective way to store rarely used data. Penn Medicine Online Data Archive is a cloud-based storage service provided by Microsoft Azure, designed for long-term retention of infrequently accessed data at a low cost. It's an ideal solution for organizations looking to store large amounts of archival data that may need to be preserved for years or even decades.

Key Features of Penn Medicine Online Data Archive:

• Cost-Effective Storage: Penn Medicine Online Data Archive offers a storage tier that is more affordable than regular cloud storage options because the data is expected to be accessed less frequently.
• Durability and High Availability: Despite its low cost, the archive tier maintains high levels of durability and availability. Data is replicated multiple times within an Azure region or across multiple regions.
• Security and Compliance: Penn Medicine Online Data Archive provides robust security measures including encryption at rest and advanced threat protection. It also complies with various industry standards and certifications which is critical for sensitive archival data.
• Scalability: The service scales automatically to accommodate growing amounts of archival data without the need for significant management overhead.

Use Cases:

• Long-Term Data Retention: For regulatory compliance where certain types of data need to be retained over a long period.
• Backup and Disaster Recovery: Storing backups in a cost-efficient way while ensuring they are available if needed.
• Big Data Analytics: Archiving large datasets that are not required for immediate analysis but might be useful in the future.

If you are interested in learning more about Penn Medicine Online Data Archive, please email pmacs_sio@pennmedicine.upenn.edu.

The updated PSOM Faculty Salary Increase 2.0 application launched a couple of months ago and was effectively used for this year's process. This update addresses outdated technology and integrates seamlessly with Workday. Developed in collaboration with Decision Support and Analysis (DSA), the new version streamlines the salary increase process with a user-friendly interface. By directly interfacing with Workday, the solution eliminates redundant data entry, reducing errors, and manual tasks.

The PSOM Translational Research Center Lab Billing 2.0 integrates tracking and reporting for Glasswash and CO2 services into its automated billing system developed by PMACS. The new application centralizes tasks previously scattered across multiple spreadsheets overseen by the PSOM Space Planning Office (SPO) into a single platform. The target launch is scheduled for the beginning of the 2025 fiscal year.

The PSOM Stockroom 2.0: Customer Catalog has been revitalized and reconstructed on the Ruby on Rails platform developed by PMACS. PSOM customers will benefit from a streamlined workflow and a user-friendly interface, enabling them to submit their 200 daily orders, resulting in 1 million dollars in monthly sales. The system is presently undergoing beta testing with selected labs, and we anticipate a complete rollout within the coming month.

The LIMS team supports many types of biobanks from internal Penn laboratories and cores to external research efforts. Two new impactful projects have gone LIVE and are now collecting samples in the LabVantage specimen tracking system.

The first Go-LIVE is the Lung Transplant Consortium (LTC) PROMISE study, which includes 19 sites from medical centers across the United States. The University of Pennsylvania is operating at the data coordinating center for this initiative and using LabVantage to label and track kits and samples distributed across these multiple sites. This initiative has helped make this external partnership possible by simplifying the processes of supporting remote labs in sample labeling and shipment and allowing one Penn Medicine group to support a large research collaboration.

The second study is the Dermatology Biobank here at Penn Medicine, with the first samples logged on July 10^th. The Dermatology Biobank leverages the broad research consent from the Penn Medicine Biobank (PMBB) and LabVantage LIMS supports visibility into sample inventories for both groups.  This effort builds upon existing PMBB workflows in LabVantage which streamlines both support of the studies and collaboration between the groups.

As of August 6^th, 2024, physicians now have the ability to leverage the power of Cosmos to enable a new feature called Look-Alikes. Cosmos is a dataset created in collaboration with a community of health systems using Epic and is designed to improve patient care. By combining their data, participating organizations and Epic can make new discoveries and advance medicine. Organizations that choose to participate in Cosmos contribute discrete patient data to the centralized Cosmos database.

Very rare conditions and medical mysteries often result in limited evidence for a physician to draw from when determining a course of action or treatment for their patient. Look-Alikes enables providers to connect with other providers across the Cosmos community who might have experience with patients with a similar constellation of symptoms or conditions so they can collaborate on care. If you are interested in learning more, contact PennChartResearch@pennmedicine.upenn.edu.

Did you know physical breaches in healthcare facilities can be as devastating as cyber-attacks? For example, during our last onsite HIPAA security review, the auditors found doors to clinical areas propped open, allowing free passage of patients or passersby into clinical areas without escort. Physical security is crucial at Penn Medicine as it protects patients, staff, sensitive information, and medical equipment.

Please remember to:

• Close doors to non-public areas.
• Check for security badges and visitor passes, particularly in non-public areas.
• Keep a clean desk and clear papers away from printers or faxes.
• Properly dispose of paper with sensitive information using shredder bins, not regular trash.