3D Printing MRI / CT / Ultrasound Data: Part 1

Support material for the 3D printed brain in Cura

Based on the 4-part tutorial where we segmented the brain from an MRI image, one of the most interesting application areas is printing such 3D models. In that sense, it makes no difference if the data is coming from an MRI (e.g., a brain or tumor), CT (e.g., the skull) or ultrasound. In this article, we’ll look at how to prepare the 3D model for 3D printing.

In the preparation phase, we segmented the model from the original DICOM medical data using 3D Slicer. Afterwards, we reduced the level of detail using the built-in tools in Windows 10.

In this part, we print the MRI brain model using the Witbox 2 3D printer with plastic and deal with support structures. The aim is to make this process accessible for everyone – so you don’t need specialized and expensive software & hardware; we’ll instead use open source and free tools as much as possible.

Special thanks to Christoph Braun from the FH St. Pölten, who is the resident 3D printing expert and prepared the steps to produce the amazing results! Continue reading “3D Printing MRI / CT / Ultrasound Data: Part 1”

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 4: Segmenting the Brain

3D Builder: show 3D model of brain segmented from MRI / MRT image

In the previous blog posts, we’ve used a simple grayscale threshold to define the model surface for visualizing a MRI / CT / Ultrasound in 3D. In many cases, you need to have more control over the 3D model generation, e.g., to only visualize the brain, a tumor or a specific part of the scan.

In this blog post, I’ll demonstrate how to segment the brain of an MRT image; but the same method can be used for any segmentation. For example, you can also build a model of the skull based on a CT by following the steps below. Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 4: Segmenting the Brain”

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 3: 3D Model Maker

MRI 3D Model visualized in Windows 10 3D Builder

So far, we’ve created a volume rendering of a MRI / CT / Ultrasound scan. This is based on Voxels. For 3D printing and highly performant visualization in AR / VR scenarios, we need to create and export a polygon-based model. For the first step, we will use the Grayscale Model Maker and export the 3D Model as .stl to further prepare the model.

To create a 3D model, we have two main options in 3D Slicer:

  • Grayscale Model Maker: directly uses grayscale values from the image data. A threshold defines the surfaces. The model maker also takes care of smoothing the surfaces and reducing the polygon count.
  • Model Maker: this requires labels or discrete data to build a 3D model, meaning you have to segment the image data.

As a first step, we will use the Grayscale Model Maker, and later explore the more advanced options offered by segmentation and the Model Maker. Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 3: 3D Model Maker”

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 2: 3D Volume Rendering

Region of Interest in a 3D Volume Rendering of an MRI in 3D Slicer

After importing the MRI / CT / Ultrasound data into 3D Slicer in part 1, we’re ready for the first 3D visualization inside the medical software through 3D Volume Rendering. This is an important step to ultimately export the 3D model to Unity for visualization through Google ARCore or Microsoft HoloLens, or for 3D printing.

Slices in 3D View

After optimizing brightness and contrast of the image data, the easiest way of showing the data in 3D is to visualize the three visible slices (planes: axial / top / red; sagittal / side / yellow; coronal / frontal / green view) in the 3D view. This gives a good overview of the position and the relation of the slices to each other. Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 2: 3D Volume Rendering”

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 1: Importing Data

MRI data with auto W/L in 3D Slicer

Some of the best showcases of Mixed Reality / VR / AR include 3D visualizations of MRI (magnetic resonance imaging), CT (computer tomography) or ultrasound scans. 3D brings tremendous advantages for analyzing the scanned images compared to only viewing 2D slices. Additionally, a good visualization brings value to patients who can gain a better understanding if they can easily explore their own body.

As part of the 3D information visualization lecture at the FH St. Pölten, I’m giving an overview of the process of converting an MRI / CT / ultrasound scan into a hologram that you can view on the Microsoft HoloLens or with Google ARCore. This blog post series explains the hands-on parts, so that you can easily re-create the same results using freely available tools. Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 1: Importing Data”

How to Combine the Mixed Reality Toolkit, Unity 2017.2 Beta and Visual Studio 2017.3

MRI scan visualized through HoloLens and Mixed Reality

Update 19. October 2017: In the meantime, Unity 2017.2 final has been released, and the dev branch of the Mixerd Reality toolkit has been merged back to the master. You should now be fine using the following versions for HoloLens development: Unity 2017.2.0f3+, Mixed Reality Toolkit (master branch), Visual Studio 2017.+4, Windows 10.0.15063.0 SDK.

Original Article: Lately, the tools required for HoloLens / Mixed Reality development have been undergoing profound changes. All three tools involved in building HoloLens apps are being restructured:

  • Unity 2017 unifies Virtual / Augmented Reality APIs, making them flexible enough to target all platforms (e.g., phones with ARKit / ARCore, VR, AR). This also involves new and renamed APIs.
  • HoloToolkit has been renamed to Mixed Reality Toolkit, as Microsoft expands the scope to include the new VR headsets with inside-out tracking going on sale this fall.
  • Visual Studio 2017.3 also introduced some major changes under the hood. This is combined with the C# engine used in Unity slowly being migrated from the old Mono runtime to more recent versions of C#.

With the latest Unity 2017.2.0b11 release, everything should now be coming together. In this blog post, I’m describing how to use the latest versions of the tools for creating and deploying a HoloLens app. Continue reading “How to Combine the Mixed Reality Toolkit, Unity 2017.2 Beta and Visual Studio 2017.3”

Windows Developer Day, Oct 10th, Vienna: Mixed Reality, Xamarin & more!

Mobile Developer After-Work

The Windows 10 Fall Creators Update brings an impressive array of new features for developers. At the Windows Developer Day, the Microsoft engineering teams will introduce the latest news for developers.

One of the most exciting is Windows Mixed Reality as a feature of the base operating system, bringing Virtual Reality to the masses with reasonably priced VR headsets. Additional topics include Xamarin, .NET Standard 2.0 and the XBox Live Creators Program.

MVPs (Microsoft Most Valuable Professionals) around the globe organize viewing parties. Through the Austrian mobility.builders mobile developer community and together with fellow MVP Helmut Krämer, I have the honor of hosting the local streaming party in Vienna. Continue reading “Windows Developer Day, Oct 10th, Vienna: Mixed Reality, Xamarin & more!”

Basics of Web Technology: HTML5, CSS, JavaScript, Accessibility & WordPress

Basics of Web Technologies

It’s important to know the basics of how the web works. In one of the introductionary lectures at Digital Healthcare, we take a look at HTML5, CSS, JavaScript & more. Understanding basic web technology is required to ensure that web sites have excellent usability as well as accessibility.

This includes how to make sure navigating the page works with screen readers, as well as that the design works well for people with e.g., (color) vision deficiency. Accessibility is especially important for websites and mobile apps of public sector bodies, based on a new directive of the EU that will come into effect in all member states soon. Continue reading “Basics of Web Technology: HTML5, CSS, JavaScript, Accessibility & WordPress”

Reading Blood Oxygen Saturation (SpO2) and Pulse through MySignals / Arduino

The MySignals HW BLE v2 – eHealth and Medical IoT Development Platform for Arduino contains sensors that measure more than 20 biometric parameters. One of the most interesting is the Bluetooth LE SpO2 sensor. How to get started reading live data and visualizing it on the TFT display of the board?

What is the MySignals HW Kit?

The MySignals HW kit contains a shield that requires an Arduino Uno as base. In contrast to the (more expensive) MySignals SW kit, the HW kit is rather basic. Powering it up results in a white screen, instead of a nice interface on the screen. The rest is up to the software developer. As the development is based on Arduino, you need the latest version of the Arduino IDE. Continue reading “Reading Blood Oxygen Saturation (SpO2) and Pulse through MySignals / Arduino”

Cross-Platform Universal Bluetooth Beacon Library Evolves

Universal Bluetooth Beacon Library

Thanks to the help of the new collaborator Chris Tacke, the Universal Bluetooth Beacon Library has evolved to its next level. Based on a modularized architecture, the library is now based on .NET Standard 1.3. Extensions provide the glue to the Bluetooth APIs of various platforms.

Currently, Windows 10 (UWP) and Android (Xamarin) are supported. The core library also works fine on iOS, Mac and Linux. Continue reading “Cross-Platform Universal Bluetooth Beacon Library Evolves”