Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 4: Segmenting the Brain

3D Builder: show 3D model of brain segmented from MRI / MRT image

In the previous blog posts, we’ve used a simple grayscale threshold to define the model surface for visualizing a MRI / CT / Ultrasound in 3D. In many cases, you need to have more control over the 3D model generation, e.g., to only visualize the brain, a tumor or a specific part of the scan.

In this blog post, I’ll demonstrate how to segment the brain of an MRT image; but the same method can be used for any segmentation. For example, you can also build a model of the skull based on a CT by following the steps below. Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 4: Segmenting the Brain”

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 3: 3D Model Maker

MRI 3D Model visualized in Windows 10 3D Builder

So far, we’ve created a volume rendering of a MRI / CT / Ultrasound scan. This is based on Voxels. For 3D printing and highly performant visualization in AR / VR scenarios, we need to create and export a polygon-based model. For the first step, we will use the Grayscale Model Maker and export the 3D Model as .stl to further prepare the model.

To create a 3D model, we have two main options in 3D Slicer:

  • Grayscale Model Maker: directly uses grayscale values from the image data. A threshold defines the surfaces. The model maker also takes care of smoothing the surfaces and reducing the polygon count.
  • Model Maker: this requires labels or discrete data to build a 3D model, meaning you have to segment the image data.

As a first step, we will use the Grayscale Model Maker, and later explore the more advanced options offered by segmentation and the Model Maker. Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 3: 3D Model Maker”

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 2: 3D Volume Rendering

Region of Interest in a 3D Volume Rendering of an MRI in 3D Slicer

After importing the MRI / CT / Ultrasound data into 3D Slicer in part 1, we’re ready for the first 3D visualization inside the medical software through 3D Volume Rendering. This is an important step to ultimately export the 3D model to Unity for visualization through Google ARCore or Microsoft HoloLens, or for 3D printing.

Slices in 3D View

After optimizing brightness and contrast of the image data, the easiest way of showing the data in 3D is to visualize the three visible slices (planes: axial / top / red; sagittal / side / yellow; coronal / frontal / green view) in the 3D view. This gives a good overview of the position and the relation of the slices to each other. Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 2: 3D Volume Rendering”

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 1: Importing Data

MRI data with auto W/L in 3D Slicer

Some of the best showcases of Mixed Reality / VR / AR include 3D visualizations of MRI (magnetic resonance imaging), CT (computer tomography) or ultrasound scans. 3D brings tremendous advantages for analyzing the scanned images compared to only viewing 2D slices. Additionally, a good visualization brings value to patients who can gain a better understanding if they can easily explore their own body.

As part of the 3D information visualization lecture at the FH St. Pölten, I’m giving an overview of the process of converting an MRI / CT / ultrasound scan into a hologram that you can view on the Microsoft HoloLens or with Google ARCore. This blog post series explains the hands-on parts, so that you can easily re-create the same results using freely available tools. Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 1: Importing Data”

Windows Developer Day, Oct 10th, Vienna: Mixed Reality, Xamarin & more!

Mobile Developer After-Work

The Windows 10 Fall Creators Update brings an impressive array of new features for developers. At the Windows Developer Day, the Microsoft engineering teams will introduce the latest news for developers.

One of the most exciting is Windows Mixed Reality as a feature of the base operating system, bringing Virtual Reality to the masses with reasonably priced VR headsets. Additional topics include Xamarin, .NET Standard 2.0 and the XBox Live Creators Program.

MVPs (Microsoft Most Valuable Professionals) around the globe organize viewing parties. Through the Austrian mobility.builders mobile developer community and together with fellow MVP Helmut Krämer, I have the honor of hosting the local streaming party in Vienna. Continue reading “Windows Developer Day, Oct 10th, Vienna: Mixed Reality, Xamarin & more!”

Cross-Platform Universal Bluetooth Beacon Library Evolves

Universal Bluetooth Beacon Library

Thanks to the help of the new collaborator Chris Tacke, the Universal Bluetooth Beacon Library has evolved to its next level. Based on a modularized architecture, the library is now based on .NET Standard 1.3. Extensions provide the glue to the Bluetooth APIs of various platforms.

Currently, Windows 10 (UWP) and Android (Xamarin) are supported. The core library also works fine on iOS, Mac and Linux. Continue reading “Cross-Platform Universal Bluetooth Beacon Library Evolves”

Porting the NFC / NDEF Library to .NET Standard

NdefDemo App of the NFC Library, now based on .NET Standard

With around 20,000 downloads, the “NFC / NDEF Library for Proximity APIs” is the most popular NFC library for C#. The library has initially been released in 2012 and has since been extended to cover many standardized as well as common NDEF use cases – also thanks to the support of the netidee open innovation platform.

Lately, the underlying platforms have evolved, and now the library has adapted. The main NFC library has now been ported to .NET Standard, which is a common baseline that makes the library compatible to even more different platforms. The new version 4.1.0 is now available under the open source LGPL license on GitHub, as well as on NuGet.

In addition to running on Windows, the library now fully supports all platforms that support .NET Core (like Linux and Mac), as well as Xamarin for Android and iOS. This makes it significantly easier to provide NFC functionality across different platforms – especially now that the iPhone also finally supports the open NDEF standard within NFC through the Apple Core NFC framework. Continue reading “Porting the NFC / NDEF Library to .NET Standard”

.NET Standard Libraries and NuGet Package Wizard

Generating NuGet Packages from Visual Studio 2017

A few days ago, Microsoft has released  .NET Standard 2.0, which is the new dreams-come-true platform for libraries. Additionally, Portable Class Libraries (PCL) have since also been deprecated. Therefore, it’s about time to port my existing libraries.

In addition, also Visual Studio 2017 Version 15.3 with full support for .NET Standard 2.0 was released, as well as the latest Windows 10 Fall Creators Update Preview SDK. So, let’s get started!

.NET Standard vs. UWP

However, it turns out that UWP doesn’t yet support .NET Standard 2.0. For the UWP platform, the latest supported .NET Standard version is still 1.4, which is considerably less powerful.

Attempting to reference a .NET Standard 2.0 library from an UWP app leads to errors, including the following that gives it away:

Upgrading the UWP project to the latest SDK Preview for the Windows 10 Fall Creators Update Build 16257 doesn’t change anything; both the SDK and the new Visual Studio 2017 version 15.3 can not yet use .NET Standard 2.0 in conjunction with UWP. According to Microsoft, that will be coming soon with the next UWP version. Continue reading “.NET Standard Libraries and NuGet Package Wizard”

How to Read the Battery Level of Bluetooth LE Devices

Microsoft Surface Dial

At the Microsoft Build Tour in Vienna, I had the opportunity to present an enhanced version of the Bluetooth Beacon talk. With the Windows 10 Creators Update, Microsoft released a massive update to the Bluetooth LE capabilities of the OS. This finally allows developers to use the full potential of modern Bluetooth scenarios, including sensors, wearables and beacons.

At Build 2017, Microsoft released three new sessions explaining the capabilities of the new APIs:

  1. Intro to Bluetooth LE Explorer
  2. Unpaired Bluetooth LE Device Connectivity
  3. Bluetooth GATT Server

Even though the sessions reference example code and blog posts that should have been released together with the session recordings, still none of that is available so far. That gave me the opportunity to explore the new APIs based on the short documentation overview page and the videos. Continue reading “How to Read the Battery Level of Bluetooth LE Devices”