Categories
3D Printing AR / VR Digital Healthcare HoloLens Image Processing Windows

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 2: 3D Volume Rendering

Region of Interest in a 3D Volume Rendering of an MRI in 3D Slicer

After importing the MRI / CT / Ultrasound data into 3D Slicer in part 1, we’re ready for the first 3D visualization inside the medical software through 3D Volume Rendering. This is a major step to export the 3D model to Unity for visualization through Google ARCore or Microsoft HoloLens, or for 3D printing.

Slices in 3D View

After optimizing brightness and contrast of the image data, the easiest way of showing the data in 3D is to visualize the three visible slices (planes: axial / top / red; sagittal / side / yellow; coronal / frontal / green view) in the 3D view. This gives a good overview of the position and the relation of the slices to each other.

Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 2: 3D Volume Rendering”
Categories
3D Printing AR / VR Digital Healthcare HoloLens Image Processing Windows

Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 1: Importing Data

MRI data with auto W/L in 3D Slicer

Some of the best showcases of Mixed Reality / VR / AR include 3D visualizations of MRI (magnetic resonance imaging), CT (computer tomography) or ultrasound scans. 3D brings tremendous advantages for analyzing the scanned images compared to only viewing 2D slices. Additionally, a good visualization brings value to patients who can gain a better understanding if they can easily explore their own body.

As part of the 3D information visualization lecture at the FH St. Pölten, I’m giving an overview of the process of converting an MRI / CT / ultrasound scan into a hologram that you can view on the Microsoft HoloLens or with Google ARCore. This blog post series explains the hands-on parts, so that you can easily re-create the same results using freely available tools.

Continue reading “Visualizing MRI & CT Scans in Mixed Reality / VR / AR, Part 1: Importing Data”
Categories
App Development AR / VR Digital Healthcare HoloLens

How to Combine the Mixed Reality Toolkit, Unity 2017 and Visual Studio 2017

MRI scan visualized through HoloLens and Mixed Reality

Update 20. December 2017: A new release of the Mixed Reality toolkit is now out as an official release. It’s recommended to use this, along with Unity 2017.2.1f1.

Update 13. November 2017: The latest source code of the Mixed Reality toolkit now combines both HoloLens and Mixed Reality headsets into a single toolkit that works with one Unity version: 2017.2.0p1 MRTP 4. It’s a special fork of Unity that is optimized for the “Mixed Reality Toolkit Preview”. A later version of Unity will hopefully combine all environments into a single release again. Read more about the environment setup at the GitHub pull request.

Update 19. October 2017: In the meantime, Unity 2017.2 final has been released, and the dev branch of the Mixed Reality toolkit has been merged back to the master. You should now be fine using the following versions for HoloLens development: Unity 2017.2.0f3+, Mixed Reality Toolkit (master branch), Visual Studio 2017.+4, Windows 10.0.15063.0 SDK.

Original Article: Lately, the tools required for HoloLens / Mixed Reality development have been undergoing profound changes. All three tools involved in building HoloLens apps are being restructured:

  • Unity 2017 unifies Virtual / Augmented Reality APIs, making them flexible enough to target all platforms (e.g., phones with ARKit / ARCore, VR, AR). This also involves new and renamed APIs.
  • HoloToolkit has been renamed to Mixed Reality Toolkit, as Microsoft expands the scope to include the new VR headsets with inside-out tracking going on sale this fall.
  • Visual Studio 2017.3 also introduced some major changes under the hood. This is combined with the C# engine used in Unity slowly being migrated from the old Mono runtime to more recent versions of C#.

With the latest Unity 2017.2.0b11 release, everything should now be coming together. In this blog post, I’m describing how to use the latest versions of the tools for creating and deploying a HoloLens app.

Continue reading “How to Combine the Mixed Reality Toolkit, Unity 2017 and Visual Studio 2017”
Categories
App Development AR / VR Events Windows

Windows Developer Day, Oct 10th, Vienna: Mixed Reality, Xamarin & more!

Mobile Developer After-Work
(c) www.annarauchenberger.com / Anna Rauchenberger – Wien, 05.04.2017 - Mobile Developer After-Work #14 // Creating Immersive Environments //

The Windows 10 Fall Creators Update brings an impressive array of new features for developers. At the Windows Developer Day, the Microsoft engineering teams will introduce the latest news for developers.

One of the most exciting is Windows Mixed Reality as a feature of the base operating system, bringing Virtual Reality to the masses with reasonably priced VR headsets. Additional topics include Xamarin, .NET Standard 2.0 and the XBox Live Creators Program.

MVPs (Microsoft Most Valuable Professionals) around the globe organize viewing parties. Through the Austrian mobility.builders mobile developer community and together with fellow MVP Helmut Krämer, I have the honor of hosting the local streaming party in Vienna.

Continue reading “Windows Developer Day, Oct 10th, Vienna: Mixed Reality, Xamarin & more!”
Categories
App Development Digital Healthcare

Basics of Web Technology: HTML5, CSS, JavaScript, Accessibility & WordPress

Basics of Web Technologies

It’s important to know the basics of how the web works. In one of the introductory lectures at Digital Healthcare, we look at HTML5, CSS, JavaScript & more. Understanding basic web technology is required to ensure that web sites have excellent usability as well as accessibility.

This includes how to make sure navigating the page works with screen readers, as well as that the design works well for people with e.g., (color) vision deficiency. Accessibility is especially important for websites and mobile apps of public sector bodies, based on a new directive of the EU that will come into effect in all member states soon.

Continue reading “Basics of Web Technology: HTML5, CSS, JavaScript, Accessibility & WordPress”
Categories
Digital Healthcare

Reading Blood Oxygen Saturation (SpO2) and Pulse through MySignals / Arduino

The MySignals HW BLE v2 – eHealth and Medical IoT Development Platform for Arduino contains sensors that measure more than 20 biometric parameters. One of the most interesting is the Bluetooth LE SpO2 sensor. How to get started reading live data and visualizing it on the TFT display of the board?

What is the MySignals HW Kit?

The MySignals HW kit contains a shield that requires an Arduino Uno as base. In contrast to the (more expensive) MySignals SW kit, the HW kit is rather basic. Powering it up results in a white screen, instead of a nice interface on the screen. The rest is up to the software developer. As the development is based on Arduino, you need the latest version of the Arduino IDE.

Continue reading “Reading Blood Oxygen Saturation (SpO2) and Pulse through MySignals / Arduino”
Categories
App Development Bluetooth Beacons Windows

Cross-Platform Universal Bluetooth Beacon Library Evolves

Universal Bluetooth Beacon Library

Thanks to the help of the new collaborator Chris Tacke, the Universal Bluetooth Beacon Library has evolved to its next level. Based on a modularized architecture, the library is now based on .NET Standard 1.3. Extensions provide the glue to the Bluetooth APIs of various platforms.

Currently, Windows 10 (UWP) and Android (Xamarin) are supported. The core library also works fine on iOS, Mac and Linux.

Continue reading “Cross-Platform Universal Bluetooth Beacon Library Evolves”
Categories
App Development NFC Windows

Porting the NFC / NDEF Library to .NET Standard

NdefDemo App of the NFC Library, now based on .NET Standard

With around 20,000 downloads, the “NFC / NDEF Library for Proximity APIs” is the most popular NFC library for C#. The library has initially been released in 2012 and has since been extended to cover many standardized as well as common NDEF use cases – also thanks to the support of the netidee open innovation platform.

Lately, the underlying platforms have evolved, and now the library has adapted. The main NFC library has now been ported to .NET Standard, which is a common baseline that makes the library compatible to even more different platforms. The new version 4.1.0 is now available under the open source LGPL license on GitHub, as well as on NuGet.

In addition to running on Windows, the library now fully supports all platforms that support .NET Core (like Linux and Mac), as well as Xamarin for Android and iOS. This makes it significantly easier to provide NFC functionality across different platforms – especially now that the iPhone also finally supports the open NDEF standard within NFC through the Apple Core NFC framework.

Continue reading “Porting the NFC / NDEF Library to .NET Standard”
Categories
App Development NFC Windows

.NET Standard Libraries and NuGet Package Wizard

Generating NuGet Packages from Visual Studio 2017

A few days ago, Microsoft has released  .NET Standard 2.0, which is the new dreams-come-true platform for libraries. Additionally, Portable Class Libraries (PCL) have since also been deprecated. Therefore, it’s about time to port my existing libraries.

In addition, also Visual Studio 2017 Version 15.3 with full support for .NET Standard 2.0 was released, as well as the latest Windows 10 Fall Creators Update Preview SDK. So, let’s get started!

.NET Standard vs. UWP

However, it turns out that UWP doesn’t yet support .NET Standard 2.0. For the UWP platform, the latest supported .NET Standard version is still 1.4, which is considerably less powerful.

Attempting to reference a .NET Standard 2.0 library from an UWP app leads to errors, including the following that gives it away:

Project '...' targets '.NETStandard,Version=v2.0'. It cannot be referenced by a project that targets 'UAP,Version=v10.0.10240'.

Upgrading the UWP project to the latest SDK Preview for the Windows 10 Fall Creators Update Build 16257 doesn’t change anything; both the SDK and the new Visual Studio 2017 version 15.3 can not yet use .NET Standard 2.0 in conjunction with UWP. According to Microsoft, that will be coming soon with the next UWP version.

Continue reading “.NET Standard Libraries and NuGet Package Wizard”
Categories
App Development Smart Home Windows

Installing Windows 10 IoT Core on a Raspberry Pi

Setting up a Raspberry Pi with Windows 10 IoT Core turned out to be more difficult than expected. In the end, there was a successful workaround – here’s how to get it running (even on non-supported SD cards).

Continue reading “Installing Windows 10 IoT Core on a Raspberry Pi”