PHP SDK & Packagist, making your life easier!

Over the past few years, dependency management tools have become a key component to successfully building high-level applications.

The number of third party libraries (with all the dependencies involved) has just insanely grown during the last few years – and expected to grow even more in the upcoming years – that managing the packages, the versions and the dependencies can become a serious nightmare for both developers and project managers. Dependency management tools therefore provide solutions to relieve them of the hassle of keeping dependencies up to date and organized so they can focus on their main mission, building awesome applications.

To make libraries easily shareable online and available for downloads, submissions, updates, bug reporting…Dependency management tools rely naturally on central repositories (we can name Maven central repository for Java, PyPI for python…and the list goes on).

As introduced in our previous posts, our Java SDK is already available on Maven, our Python SDK is also available on PyPI, so the next move logically is to add the PHP SDK to a central repository for PHP to make it easily available for the public…but what do we have for PHP?

For PHP, the dependency management tool to go for is composer, as pointed out earlier, it relies – like other dependency management tools – on its own central repository Packagist.

Installing Composer

In the same logic as most dependency management tools , Composer is available via command line interface which offers indeed way more powerful possibilities than a GUI.
To be able to execute Composer commands, you need to have it installed first on your computer, for more information you can refer to install composer.

Once composer is successfully installed, you can check if your environment is well set up via the following command :

Your first sample code

Next step is to create a sample code to call method getVersionInfo() from Class Core on pma-php (once obviously installed via composer).

To install pma-php via composer in a PHP project, you simply issue the following command in the project folder :

As you could already notice, we didn’t specify any version to install for composer, so it installed automatically the latest release (v2.0.0.40).

Targeting a specified release is straightforward via this command :

For the list of available releases for pma-php library, you can refer to its official Packagist page

Once the library pma-php is successfully installed on a project folder, the following files/folders are created :

Next step is to create a PHP file to call method getVersionInfo() of Class Core on pma-php :


<?php

require_once 'vendor/autoload.php';
use Pathomation\PmaPhp\Core;

echo Core::getVersionInfo();

Running this script returns value “2.0.0.1346”, which is the version of PMA.start installed on my own computer.

That’s all it takes! easy, isn’t it?

Run & share your Python code online with Jupyter notebooks

If you’ve had a look on previous posts on our Python SDK, you have a already a pretty good idea about scripts you can run against PMA.start (or PMA.core) for retrieving image data, Slide visualization

Most of the times running code locally is all what’s needed, however there are situations where being able to run it online and share it with collaborators (students, colleagues…) on the other side of the globe can come very handy and offer a richer and more interactive experience.
There are couple of free/paid services to run & share Python code online, our focus in this tutorial will be on Azure notebooks powered by Jupyter.

What are Azure notebooks & Jupyter?

As stated on the official documentation, Azure Notebooks is a free service for anyone to develop and run code in their browser using Jupyter. Jupyter is an open source project that enables combing markdown prose, executable code, and graphics onto a single canvas.

Azure Notebooks currently supports Python 2, Python 3, R and F# and their popular packages (e.g for Python the Anaconda distro is preinstalled), but our focus for the moment will be on Python 3 since it’s the minimum required version for the Python SDK

Your first Azure notebooks project

To create your first Azure notebooks project, navigate to the home page
click on Try it now then login (any Microsoft, Gmail,…. email address).

Once successfully logged in, navigate to section My Projects

Click on button New Project :

Introduce a Project name and and ID then click on Create (It’s worth mentioning that you have to set the project as Public if you wish to be able to share your notebooks with other users)

Click on the newly created project :

Notebooks are organized by projects, this makes it very efficient to create separate projects depending on scripts, target hosts and target audience to share notebooks with.

To create a new notebook, click on menu then select Notebook. Add a name for the notebook and select your Python 3 version (either 3.5 or 3.6 as both compatible with the Python SDK)

Notebooks are organized into cells, each cell can contain one or multiple python scripts to execute.

First cell should always be the following one as it’s required to install first pma_python packages on the running server before being able to interact with the Python SDK.

From there on you can add as many cells as you wish to interact with the Python SDK via scripts we introduced on previous posts or ones you create yourself.

Once your notebook modified and saved, you can easily share the created project with other users via share button on My projects page.

The terminal

In addition to its intuitive and easy-to-use interface, Azure notebooks do provide provides access to a complete terminal running on the server. To access the terminal first click on the Jupyter icon in the upper left hand corner of your notebook server. Then click on the New button on the upper right hand side of the notebook list. Finally click Terminal.

Your newly created notebook is accessible on the running server and executing shell commands there which can be useful for downloading data, copying files, inspecting processes, or editing files with traditional Unix tools.

Building digital pathology web apps with Java

Working with slides

The Java Core class is part of our PMA.java API (introductory materials available here) and comes with a number of methods to navigate WSI slides on your local hard disk. Most often you’ll be alternating between getDirectories() and getSlides().

Here’s an example that will allow you to navigate your hard disk in a tree-like fashion :


import java.io.IOException;
import java.net.URLEncoder;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;

public class Test extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      ServletOutputStream out = response.getOutputStream();

      if (!Core.isLite()) {
         System.out.println("PMA.start not found");
         return;
      }

      Core.connect();

      out.println("<html>");
      out.println("<ul>");
      if (request.getParameter("p") == null) {
         for (String rd : Core.getRootDirectories()) {
            out.println(
                  "<li><a href='?p=" + URLEncoder.encode(rd, "UTF-8").replace("+", "%20") + "'>" + rd + "</li>");
         }
      } else {
         String[] parts = request.getParameter("p").split("/");
         for (String rd : Core.getRootDirectories()) {
            if (parts[0].equals(rd)) {
               out.println("<li><b>" + rd + "</b>");
               for (String subdir : Core.getDirectories(rd)) {
                  out.println("<ul>");
                  String subdirparts[] = subdir.split("/");
                  if (request.getParameter("p").indexOf(subdir) == -1) {
                     out.println("<li><b>" + subdirparts[subdirparts.length - 1] + "</b>");
                     // keep drilling down, or see if you can retrieve slides as well
                     out.println("</li>");
                  } else {
                     out.println("<li><a href='?p=" + URLEncoder.encode(subdir, "UTF-8").replace("+", "%20")
                           + "'>" + subdirparts[subdirparts.length - 1] + "</a></li>");
                  }
                  out.println("</ul>");
               }
               out.println("</li>");
            } else {
               out.println("<li><a href='?p=" + URLEncoder.encode(rd, "UTF-8").replace("+", "%20") + "'>" + rd
                     + "</a></li>");
            }
         }
      }
      out.println("</ul>");
      out.println("</html>");
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

Yes, this should all be in a recursive function so you can dig down to just about any level in your tree structure. However, this post is not about recursive programming; we’re mostly interested in showing you what our API/SDK can do.

For instance, you can retrieve the slides in a selected folder and generate a link to them for visualization:


// now handle the slides in the subfolder
out.println("<ul>");
for (String slide : Core.getSlides(subdir)) {
   String[] slideParts = slide.split("/");
   out.println("<li>" + slideParts[slideParts.length - 1] + "</li>");
}
out.println("</ul>");


Introducing the UI class

We can do better than our last example. Providing a link to PMA.start is easy enough, but once you get to that level you’ve lost control over the rest of your layout. What if you make a website where you want to place slides in certain predefined locations and placeholders?

That’s where the UI class comes in. Currently, you can use it to either embed slide viewports, or thumbnail galleries in your own website.

Here’s how you can include an arbitrary slide:


import java.io.IOException;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;
import com.pathomation.UI.UI;

public class Test extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      // setup parameters
      UI.pmaUIJavascriptPath = UI.pmaStartUIJavascriptPath;
      ServletOutputStream out = response.getOutputStream();
      String sessionID = Core.connect();

      // pick a slide to embed in your page
      String slide = Core.getSlides("C:/my_slides/").get(0);
      List<String> results = UI.embedSlideBySessionID("http://localhost:54001/", slide, sessionID);
      // the first element in the list contains the generated front end code
      out.println(results.get(0));
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}


The embedSlideBySessionID() method return a string that serves as an identifier for the generated viewport. Use this identifier to subsequently define a style for your viewport:


// actually embed slide
// the second element in the list corresponds to the viewport ID
String viewport = UI.embedSlideBySessionID("http://localhost:54001/", slide, sessionID).get(1);
out.println("<style type=\"text/css\">\n" + 
"#" + viewport + "{\n" 
      + "width: 500px;\n" 
      + "height: 500px;\n"
      + "border: 2px dashed green;\n" 
      + "}\n" 
+ "</style>");

The result is now a 500 x 500 fixed square (with a dashed border) that doesn’t change as your modify the browser window:


You can have as many viewports on a single page as you want; each is automatically assigned a new ID, and so you can set separate layout for each one.

Working with galleries

What if you have a collection of slides and you want to present an overview of these (browsing through slide filenames is tiring and confusing). You could already combine the code we have in this post so far and request thumbnails for a list of a slides found in a directory, subsequently rendering selected slides in a viewport.

But what if you have 50 slides in the folder? Do you really want to handle the scrolling, just-in-time rendering of initially hidden thumbnails etc.?

Pretty early on in our Pathomation career we found ourselves facing the same problems. We re-invented our own wheel a couple of times, after which we figured it was round enough to encapsulate in a piece of re-usable code.

You guessed it: the UI class provides a way to generate galleries, too. At its simplest implementation, only one line of code is needed (setup not included):


import java.io.IOException;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;
import com.pathomation.UI.UI;

public class Test extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

             ServletOutputStream out = response.getOutputStream();
             String sessionID = Core.connect();
             out.println("<p>" + sessionID + "</p>\n");
             
             UI.pmaUIJavascriptPath = UI.pmaStartUIJavascriptPath;

             List<String> results = UI.embedGalleryBySessionID("http://localhost:54001/",
             "C:/my_slides", sessionID);
             // the first element in the list contains the generated front end code
             out.println(results.get(0));        
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

You’ll notice that you can select slides in the gallery, but they’re not particularly reactive. For that, you’ll need to instruct PMA.UI to provide a viewport as well. When a slide is clicked in the gallery, the slide is then shown in the viewport:

out.println(UI.linkGalleryToViewport(gallery, "viewer"));

 

The default orientation of a gallery is “horizontal”, but you can set it to a vertical layout, too:


List<String> results = UI.embedGalleryBySessionID("http://localhost:54001/", "C:/my_slides", sessionID,
      new HashMap<String, String>() {
         {
            put("mode", "vertical");
         }
      });
// the first element in the list contains the generated front end code
out.println(results.get(0));

In which you can build something that looks like this:

 

Try it!

You can build pretty complicated interfaces already this way. One possibly scheme e.g. is where you offer end-users the possibility to compare slides. You need two galleries and two viewports, and that goes like this:


List<String> results = UI.embedGalleryBySessionID("http://localhost:54001/", "C:/my_slides", sessionID);
List<String> results2 = UI.embedGalleryBySessionID("http://localhost:54001/", "C:/my_slides", sessionID);
try {
	out.println(
		 "<table width=\"100%\"><tr><th width=\"50%\">Slide 1</th><th "
		 + "width=\"50%\">Slide 2</th></tr><tr><td width=\"50%\" valign=\"top\">"
		 + results.get(0)
		 + "</td><td width=\"50%\" valign=\"top\">"
		 + results2.get(0)
		 + "</td></tr><tr><td width=\"50%\" valign=\"top\">"
		 + UI.linkGalleryToViewport(results.get(1), "viewerLeft")
		 + "</td><td width=\"50%\" valign=\"top\">"
		 + UI.linkGalleryToViewport(results2.get(1), "viewerRight")
		 + "</td></tr></table>");	
} catch (Exception e) {
	e.printStackTrace();
}

Under the hood

As demonstrated in this post through the Java SDK : simple single-line instructions in Java are translated in whole parts of JavaScript code. The UI class takes care of loading all the required libraries, and makes sure housekeeping is taken care of in case of multiple viewports, galleries, etc.

You can see this for yourself by looking at the source-code of your page, after it loads in the webbrowser.

The JavaScript framework that we wrote ourselves for browser-based WSI visualization is called PMA.UI. It comes with its own set of tutorials, and there is much more you can do with PMA.UI than through the Java SDK alone.

However, we found in practice that there is much demand for cursive embedding of WSI content in any number of places on a website or corporate intranet. In my cases, a gallery browser and a “live” slide viewport are sufficient. In those scenarios, the Java SDK can definitely come in handy and offer a reduced learning curve.

The SDK should help you get started . By studying the interaction between the Java-code and the generated JavaScript, you can eventually master the PMA.UI library as well and interact with it directly.

By all means, do send us screenshots of your concoctions (let us know when you need help from your friendly neighborhood pathologists, too)! Perhaps we can have a veritable “wall of WSI fame” up one day.

Code samples for the Java SDK

Java sample code availability

Three months ago we introduced the SDK for Java.

A set of code samples can come very handy to ease end users’ first interaction with both PMA.Start and its bigger brother PMA.core. A folder named “samples” was added for this purpose to our PMA.java GitHub repository.

Introduction

For the samples we added, we create a Java HTTPServlet class that interacts with the Java SDK to generate HTML code to be displayed on a web browser.
We aim to make the samples as simple & general as possible, end users can then adapt them to their needs depending on what architecture & frameworks they rely on for their Java web application.

PMA.Start

For the following samples to be run successfully, you’ll need to have PMA.Start installed and running on your computer.
For more information about PMA.Start, refer to https://free.pathomation.com
If you don’t have PMA.Start installed yet on your computer, make sure you download it from https://free.pathomation.com/download/ for free.

This first sample checks if PMA.Start (PMA.core.lite) is installed and running on end user’s computer :


package samples_10;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.*;

public class IdentifyingPMAStart extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      ServletOutputStream out = response.getOutputStream();
      out.println("<html>");
      // test for PMA.core.lite (PMA.start)
      out.println("Are you running PMA.core.lite? " + (Core.isLite() ? "Yes!" : "no :-(") + "<br />");
      out.println("Seeing 'no' and want to see 'yes'? Make sure PMA.start is running on your system or download it from "
                  + "<a href = \"http://free.pathomation.com\">http://free.pathomation.com</a>");
      out.println("</html>");

   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

If PMA.Start is actually running, it returns :

Version info

The second sample is quite similar to the previous one for it checks if PMA.Start is running, if yes it displays its version number

package samples_20;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;

public class GettingVersionInformationPMAStart extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      ServletOutputStream out = response.getOutputStream();
      out.println("<html>");
      // test for PMA.core.lite (PMA.start)
      if (!Core.isLite()) {
         // don't bother running this script if PMA.start isn't active
         out.println("PMA.start is not running. Please start PMA.start first");
      } else {
         // assuming we have PMA.start running; what's the version number?
         out.println("You are running PMA.start version " + Core.getVersionInfo());
      }
      out.println("</html>");
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

Making the connection

For the third sample, a connection is established to PMA.Start to retrieve a sessionID


package samples_30;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;

public class ConnectingToPMAStart extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
         
      ServletOutputStream out = response.getOutputStream();
      String sessionID = Core.connect();
      out.println("<html>");
      if (sessionID == null) {
         out.println("Unable to connect to PMA.start");
      } else {
         out.println("Successfully connected to PMA.start; sessionID = " + sessionID);
         // not always needed; depends on whether the client (e.g. browser) still needs to SessionID as well
         Core.disconnect();
      }
      out.println("</html>");   
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

If the connection succeeds, the sessionID is then displayed on the web browser

Finding your hard disks

For the fourth sample, a connection is established to PMA.Start, then the local HDD is parsed to retrieve root directories (Drive letters) :


package samples_40;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;

public class GettingDriveLettersFromPMAStart extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      ServletOutputStream out = response.getOutputStream();
      out.println("<html>");
      // establish a default connection to PMA.start
      if (Core.connect() != null) {
         out.println("The following drives were found on your system:" + "<br/>");
         for (String rootDirectory : Core.getRootDirectories()) {
            out.println(rootDirectory + "<br/>");
         }
         out.println("Can't find all the drives you're expecting? For network-connectivity (e.g. mapped drive access) you need PMA.core instead of PMA.start");
      } else {
         out.println("Unable to find PMA.start");
      }
      out.println("</html>");         
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

If the connection succeeds, a list of drive letters is displayed one by one on the web browser

Navigating directories

The fifth sample is very similar to previous one, it selects the first root directory (drive letter), its content is then displayed one by one


package samples_60;

import java.io.IOException;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;

public class GettingDirectoriesPMAStart extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      ServletOutputStream out = response.getOutputStream();
      out.println("<html>");
      String sessionID = Core.connect();
      if (sessionID == null) {
         out.println("Unable to connect to PMA.start");
      } else {
         out.println("Successfully connected to PMA.start" + "<br/>");
         List<String> rootDirs = Core.getRootDirectories();
         out.println("Directories found in " + rootDirs.get(0) + ":" + "<br/>");
         List<String> dirs = Core.getDirectories(rootDirs.get(0), sessionID);
         for (String d : dirs) {
            out.println(d + "<br/>");
         }
         // not always needed; depends on whether the client (e.g. browser) still needs to SessionID as well
         Core.disconnect(sessionID);
      }
      out.println("</html>");         
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

Successful connection results in a list of drive letters with their respective folders shown one by one

Looking for slides

For the next sample, a connection is established to PMA.Start, then we look for the first non empty directory inside the local HDD to browse its content


package samples_80;

import java.io.IOException;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;

public class GettingSlidesPMAStart extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      ServletOutputStream out = response.getOutputStream();
      out.println("<html>");
      String sessionID = Core.connect();
      if (sessionID == null) {
         out.println("Unable to connect to PMA.start");
      } else {
         out.println("Successfully connected to PMA.start" + "<br/>");
         // for this demo, we don't know where we can expect to find actual slides
         // the getFirstNonEmptyDirectory() method wraps around recursive calls to getDirectories() and is useful to "just" find a bunch of slides in "just" any folder
         String dir = Core.getFirstNonEmptyDirectory("/", sessionID);
         out.println("Looking for slides in " + dir + ":" + "<br/>");
         List<String> slides = Core.getSlides(dir, sessionID);
         for (String slide : slides) {
            out.println(slide + "<br/>");
         }
         // not always needed; depends on whether the client (e.g. browser) still needs to SessionID as well
         Core.disconnect(sessionID);
      }
      out.println("</html>");         
      
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

A quite similar result can be obtained if connection succeeds

Listing slides

Last sample for PMA.Start is a continuation for previous one, once the first non empty directory is found, the list of slides within is displayed successively (besides each slide name, its UID is added)


package samples_90;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;

public class GetUIDSlidePMAStart extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      ServletOutputStream out = response.getOutputStream();
      out.println("<html>");
      String sessionID = Core.connect();
      if (sessionID == null) {
         out.println("Unable to connect to PMA.start");
      } else {
         out.println("Successfully connected to PMA.start" + "<br/>");
         String dir = Core.getFirstNonEmptyDirectory("/", sessionID);
         out.println("Looking for slides in " + dir + "<br/>");
         for (String slide : Core.getSlides(dir, sessionID)) {
            out.println(slide + " - " + Core.getUid(slide, sessionID) + "<br/>");
         }
         // not always needed; depends on whether the client (e.g. browser) still needs to SessionID as well
         Core.disconnect(sessionID);
      }
      out.println("</html>");      
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

Here is what you can get for a successful connection

PMA.core

One main difference between PMA.Start and PMA.core is the former is run on a server which implies end users connect to through authentication (server URL, username, password).
For better code organisation & readability, we added Config class which intends to provide a single file to define the connection values to interact with PMA.core.
As You can notice later on the samples for PMA.core, HTTPServlet Classes for each sample retrieve connection values from Config class.
End users are invited to substitute the server’s url & credentials with their own respective values!


package configuration;

public class Config {
   
   // modify the following three lines for your specific circumstances:
   public static String pmaCoreServer = "http://my_server/pma.core";
   public static String pmaCoreUser = "user";
   public static String pmaCorePass = "secret";
}

First sample on the list checks if the url provided refers to a PMA.Start instance


package samples_10;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.*;
import Configuration.Config;

public class IdentifyingPMACore extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      String pmaCoreServer = Config.pmaCoreServer;

      ServletOutputStream out = response.getOutputStream();
      out.println("<html>");
      // testing actual "full" PMA.core instance that may or may not be out there
      out.println("Are you running PMA.start(PMA.core.lite) at " + pmaCoreServer + " ? " + ((Core.isLite(pmaCoreServer) != null && 
            (Core.isLite(pmaCoreServer) == true))  ? "Yes!" : "no :-(") + "<br />");
      out.println(
            "Are you running PMA.start(PMA.core.lite) at http://nowhere ? "
                  + ((Core.isLite("http://nowhere") != null && (Core.isLite("http://nowhere") == true)) ? "Yes!" : "no :-("));
      out.println("</html>");

   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

The second sample retrieves the version info for the provided PMA.core instance, it also shows what you would get trying a “bogus” URL


package samples_20;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;
import Configuration.Config;

public class GettingVersionInformationPMACore extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
         
      String pmaCoreServer = Config.pmaCoreServer;
      
      ServletOutputStream out = response.getOutputStream();
      out.println("<html>");
      out.println("You are running PMA.core version " + Core.getVersionInfo(pmaCoreServer) + " at " + pmaCoreServer + "<br/>");
      
      // what happens when we run it against a bogus URL?
      String version = Core.getVersionInfo("http://nowhere/");
      
      if (version == null) {
         out.println("Unable to detect PMA.core at specified location (http://nowhere/)");
      } else {
         out.println("You are running PMA.core version " + version);
      }
      out.println("</html>");
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

For the third sample a connection is established to the PMA.Core instance, then the sessionID generated is displayed on web browser


package samples_30;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;
import Configuration.Config;

public class ConnectingToPMACore extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
         
      String pmaCoreServer = Config.pmaCoreServer;
      String pmaCoreUser = Config.pmaCoreUser;
      String pmaCorePass = Config.pmaCorePass;
      
      ServletOutputStream out = response.getOutputStream();
      String sessionID = Core.connect(pmaCoreServer, pmaCoreUser, pmaCorePass);
      out.println("<html>");
      if (sessionID == null) {
         out.println("Unable to connect to PMA.core at specified location (" + pmaCoreServer + ")");
      } else {
         out.println("Successfully connected to PMA.core; sessionID = " + sessionID);
         // not always needed; depends on whether the client (e.g. browser) still needs to SessionID as well
         Core.disconnect();
      }
      out.println("</html>");   
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

In analogy to previous PMA.Start samples, The next sample searches for root directories of the provided PMA.core instance and displays them one by one


package samples_40;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;
import Configuration.Config;

public class GettingRootDirectoriesFromPMACore extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      String pmaCoreServer = Config.pmaCoreServer;
      String pmaCoreUser = Config.pmaCoreUser;
      String pmaCorePass = Config.pmaCorePass;
      
      ServletOutputStream out = response.getOutputStream();
      String sessionID = Core.connect(pmaCoreServer, pmaCoreUser, pmaCorePass);
      out.println("<html>");
      if (sessionID == null) {
         out.println("Unable to connect to PMA.core at specified location (" + pmaCoreServer + ")");
      } else {
         out.println("Successfully connected to " + pmaCoreServer + "<br/>");
         out.println("You have the following root-directories at your disposal:" + "<br/>");
         for (String rd : Core.getRootDirectories(sessionID)) {
            out.println(rd + "<br/>");
         }
         Core.disconnect(sessionID);
      }
      out.println("</html>");         
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

The fifth sample on the list is very similar to previous one, except that instead of displaying a complete list of root directories it only displays the first one and browses its content


package samples_60;

import java.io.IOException;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;
import Configuration.Config;

public class GettingDirectoriesPMACore extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      String pmaCoreServer = Config.pmaCoreServer;
      String pmaCoreUser = Config.pmaCoreUser;
      String pmaCorePass = Config.pmaCorePass;

      ServletOutputStream out = response.getOutputStream();
      String sessionID = Core.connect(pmaCoreServer, pmaCoreUser, pmaCorePass);
      out.println("<html>");
      if (sessionID == null) {
         out.println("Unable to connect to PMA.core at specified location (" + pmaCoreServer + ")" + "<br/>");
      } else {
         out.println("Successfully connected to PMA.core; sessionID = " + sessionID + "<br/>");
         List<String> rootDirs = Core.getRootDirectories();
         out.println("Directories found in " + rootDirs.get(0) + ":" + "<br/>");
         List<String> dirs = Core.getDirectories(rootDirs.get(0), sessionID);
         for (String d : dirs) {
            out.println(d + "<br/>");
         }
         // not always needed; depends on whether the client (e.g. browser) still needs to SessionID as well
         Core.disconnect(sessionID);
      }
      out.println("</html>");         
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

For the next sample, a connection is established to the provided PMA.core instance.
If Successful, we look for the first non empty directory to display the list of slides within successively


package samples_80;

import java.io.IOException;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;
import Configuration.Config;

public class GettingSlidesPMACore extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      String pmaCoreServer = Config.pmaCoreServer;
      String pmaCoreUser = Config.pmaCoreUser;
      String pmaCorePass = Config.pmaCorePass;
      
      ServletOutputStream out = response.getOutputStream();
      String sessionID = Core.connect(pmaCoreServer, pmaCoreUser, pmaCorePass);
      out.println("<html>");
      if (sessionID == null) {
         out.println("Unable to connect to PMA.core at specified location (" + pmaCoreServer + ")" + "<br/>");
      } else {
         out.println("Successfully connected to PMA.core; sessionID = " + sessionID + "<br/>");
         // for this demo, we don't know where we can expect to find actual slides
         // the getFirstNonEmptyDirectory() method wraps around recursive calls to getDirectories() and is useful to "just" find a bunch of slides in "just" any folder
         String dir = Core.getFirstNonEmptyDirectory("/", sessionID);
         out.println("Looking for slides in " + dir + ":" + "<br/>");
         List slides = Core.getSlides(dir, sessionID);
         for (String slide : slides) {
            out.println(slide + "<br/>");
         }
         // not always needed; depends on whether the client (e.g. browser) still needs to SessionID as well
         Core.disconnect(sessionID);
      }
      out.println("</html>");      
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

What we did on previous sample is extended on the last sample on this list, so in addition to displaying slides’ name we add also the UID for each slide


package samples_90;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.pathomation.Core;
import Configuration.Config;

public class GetUIDSlidePMACore extends HttpServlet {

   private static final long serialVersionUID = 1L;

   @Override
   protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {

      String pmaCoreServer = Config.pmaCoreServer;
      String pmaCoreUser = Config.pmaCoreUser;
      String pmaCorePass = Config.pmaCorePass;
      
      ServletOutputStream out = response.getOutputStream();
      String sessionID = Core.connect(pmaCoreServer, pmaCoreUser, pmaCorePass);
      out.println("<html>");
      if (sessionID == null) {
         out.println("Unable to connect to PMA.core at specified location (" + pmaCoreServer + ")" + "<br/>");
      } else {
         out.println("Successfully connected to PMA.core; sessionID = " + sessionID + "<br/>");
         String dir = Core.getFirstNonEmptyDirectory("/", sessionID);
         out.println("Looking for slides in " + dir + "<br/>");
         for (String slide : Core.getSlides(dir, sessionID)) {
            out.println(slide + " - " + Core.getUid(slide, sessionID) + "<br/>");
         }
         // not always needed; depends on whether the client (e.g. browser) still needs to SessionID as well
         Core.disconnect(sessionID);
      }
      out.println("</html>");      
   }

   @Override
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
         throws ServletException, IOException {
      this.doGet(request, response);
   }
}

What’s next?

In this blog post we toured you around our PMA.java SDK. We focused on basic back-end interactions like connecting and finding out which slides are available. We showed you how to navigate around our sample files that we now deposit as part our respective github repositories.

We intend to keep building on top of these initial samples. At the time you read this, there may already be more samples available in addition to the ones described here, so make sure to check.

In a next blog post, we intend to provide you with sample code to start building graphical user interfaces for digital pathology using the pathomation.UI namespace (if you just can’t wait; feel free to already read how this is done in PHP).

Our SDK is now available in Java, too!

SDK for Java

Following the release of our SDK for python, our next step is to implement the same set of functionalities for Java.
Similarly to what we already did for Python, this Java package comes with a set of methods to do basic tasks such as obtaining lists of slides, navigating a hierarchical folder structure, and, of course, extracting tiles. The code was deposited in GitHub.

To make using the SDK easier, we submitted the package to the Maven central repository

Getting an interface to Pathomation software in Java is now as easy as including this dependency tag into your pom.xml (version tag set to always retrieve latest version) :

<dependency>
    <groupId>com.pathomation</groupId>
    <artifactId>pma-java</artifactId>
    <version>[2.0,)</version>
</dependency>

You should then be able to invoke Core methods :

Getting started

What can you do with our Java SDK today? If you have PMA.start installed on your system, you can go right ahead and try out the following code:

import com.pathomation.*;

public class Test {

    public static void main(String[] args) throws Exception {
        if (Core.pmaIsLite()) {
            System.out.print("Congratulations; PMA.start is running on your system");
            System.out.print("You’re running PMA.core.lite version " + core.getVersionInfo());
        } else {
            System.out.print("PMA.start not found. Either you don’t have it installed, or you don’t have the server-component running currently");
            throw new Exception("PMA.start not detected");   
        }
    }
}

You can use the same isLite() method by the way to ask your end-user to make sure PMA.start IS running before continuing the script:

import com.pathomation.*;

public class Test {

    public static void main(String[] args) {
        if (Core.pmaIsLite()) {
            System.out.print("PMA.core.lite is NOT running.");
            System.out.print("Make sure PMA.core.lite is running and press <enter> to continue");
            System.in.read();
        }
        if (Core.pmaIsLite()) {
            System.out.print("PMA.core.lite is NOT running.");
            System.exit(1);
        }
    }
}

Slides

Now that you know how to establish the availability of PMA.start as a back-end for whole slide imaging (WSI) data, you can start looking for slides:

import com.pathomation.*;

public class Test {

    public static void main(String[] args) throws Exception {
        if (!Core.pmaIsLite()) {
            throw new Exception("PMA.start not detected");
        }
        // assume that you have slides in C:\my_slides (note the capital C)
        for (String slide : Core.getSlides("C:/my_slides")) {
            System.out.print(slide);
        }
    }
}

But you knew already that you had slides in that folder, of course. By, the way, if NO data shows up, check the specified path. It’s case sensitive, and drive letters have to be capitalized. Also make sure to use a forward slash instead of the more traditional (on Windows at least) backslash.

Now what you probably didn’t know yet is the dimensions of the slide, both in pixels as well as micrometers.

System.out.print("Pixel dimensions of slide:");
Integer xDimPix = Core.getPixelDimensions(slide).get(0),
        yDimPix = Core.getPixelDimensions(slide).get(1);
System.out.print(xDimPix.toString() + " x " + yDimPix.toString());

System.out.print("Slide surface area represented by image:");
Float xDimPhys = Core.getPhysicalDimensions(slide).get(0),
      yDimPhys = Core.getPhysicalDimensions(slide).get(1);
System.out.print(xDimPhys.toString() + "µm x " + yDimPhys.toString() + "µm = ");
System.out.print((xDimPhys * yDimPhys / 1E6) + " mm2");

Below is the output on our computer, having 3 3DHistech MRXS slides in the c:\my_slides folder. You can use this type of output as a sanity check, too.

While the numbers in µm seems huge, they start to make more sense once translated to the surface area captured. As a reminder: 1 mm2 = 1,000,000 µm2, which explains why we divide by 1E6 to get the area in mm2. 1020 mm2 still not saying much? Then keep in mind that 100 mm2 equals 1 cm2, and that 10 cm2 can very will constitute a 2 cm x 5 cm piece of tissue. A physical slide’s dimensions are typically 10 cm x 4 cm. Phew, glad the data matches reality!

Determining a slide’s magnification

We can also determine the magnification at which an image was registered. The getMagnification function has a Boolean exact= parameter that works as follows: when set to True, getMagnification will round to the nearest “whole number” magnification that’s typically mentioned on a microscope’s objective lens. This could be 5X, 20X, 40X… But bear in mind that when a microscopist looks through his device, he can fine-focus on a sample, thereby slightly modifying the actual magnification used, perhaps from 40X to 38X (even though the label on the lens still says 40X of course). Scanners work in the same manner; because of auto-focusing, the end-result of a scan may be in 38X instead of 40X, or 21X instead of 20X. And this is the number that is returned when the exact= parameter is set to True.

 

Of course, when building a Dataframe (using library TableSaw), you might as well include columns for both measurements (perhaps using the rounded measurement later for a classification task):

import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;

import com.pathomation.*;
import tech.tablesaw;

public class Test {

    public static void main(String[] args) throws Exception {

        if (!Core.pmaIsLite()) {
            throw new Exception("PMA.start not detected");
        }
        // create blank list (to be converted into a pandas DataFrame later)
        List<Map<String, Object>> slideInfos = new ArrayList<Map<String, Object>>();

        //Initialize the components for TableSaw
        StringColumn colSlide = StringColumn.create("slide");
        DoubleColumn colApproxMag = DoubleColumn.create("approxMag");
        DoubleColumn colExactMag = DoubleColumn.create("exactMag");
        BooleanColumn colIsFluo = BooleanColumn.create("isFluo");
        BooleanColumn colIsZStack = BooleanColumn.create("isZStack");
        // assume that you have slides in C:\my_slides (note the capital C)
        for (String slide : Core.getSlides("C:/my_slides")) {
            colSlide.append(Core.getSlideFileName(slide));
            colApproxMag.append(Core.getMagnification(slide, false));
            colExactMag.append(Core.getMagnification(slide, true));
            colIsFluo.append(Core.isFluorescent(slide));
            colIsZStack.append(Core.isZStack(slide));
        }
        Table results = Table.create("Results", colSlide, colApproxMag, colExactMag, colIsFluo, colIsZStack);
        results.print();
    }
}

The output of this script on our computer is as follows:

Note that for one slide, both the exact and the approximate magnification is 0. This is because that particular slide is a .jpg-file, which doesn’t contain any useful (pixels per micron) metadata to use to determine the magnification.

Representation

PMA.start is a free desktop viewer for whole slide images. Earlier, we introduced you to Core java, a novel package that serves as a wrapper-library and helps interface with PMA.start’s back-end API.

The images PMA.start typically deals with are called whole slide images, so how about we show some pixels? As it turns out, this is really easy. Just invoke the showSlide() call. Assuming you have a slide at c:\my_slides\alk_stain.mrxs, we get:

import Core.java;

public class Test {

    public static void main(String[] args)  {
        
        String slide = "C:/my_slides/alk_stain.mrxs";
        Core.showSlide(slide);
    }
}

The result depends on whether you’re using PMA.start or a full version of PMA.core. If you’re using PMA.start, you’re taken to the desktop viewer:

If you’re using PMA.core, you’re presented with an interface with less frills: the webbrowser is still involved, but nothing more than scaffolding code around a PMA.UI.View.Viewport is offered (which actually allows for more powerful applications):

Associated images

But there’s more to these images; if you only wanted to view a slide, you wouldn’t bother with Java in the first place. So let’s see what else we can get out of these?

Assuming you have a slide at c:\my_slides\alk_stain.mrxs, you can execute the following code to obtain a thumbnail image representing the whole slide:

import java.awt.BorderLayout;
import java.awt.Image;
import javax.swing.ImageIcon;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;

import com.pathomation.*;

public class Test {

    public static void main(String[] args) {

        String slide = "C:/my_slides/alk_stain.mrxs";
        Image thumb = Core.getThumbnailImage(slide);
        displayImage(thumb);
    }

    public static void displayImage(Image image) {
        JFrame frame = new JFrame();
        JLabel lblImage = new JLabel(new ImageIcon(image));
        JPanel mainPanel = new JPanel(new BorderLayout());
        mainPanel.add(lblImage);
        frame.add(mainPanel);
        frame.setVisible(true);
    }
}

But this thumbnail presentation alone doesn’t give you the whole picture. You should know that a physical glass slide usually consists of two parts: the biggest part of the slide contains the specimen of interest and is represented by the thumbnail image. However, near the end, a label is usually pasted on with information about the slide: the stain used, the tissue type, perhaps even the name of the physician. More recently, oftentimes the label has a barcode printed on it, for easy and automated identification of a slide. The label is therefore sometimes also referred to as “barcode”. Because the two terms are used so interchangeably, we decided to support them in both forms, too. This makes it easier to write code that not only syntactically makes sense, but also applies semantically in your work-environment.

A systematic representation of a physical glass slide can then be given as follows:

The Core java library then has three methods to obtain slide representations, two of which are aliases of one another:

core.getThumbnailImage() returns the thumbnail image

core.getLabelImage() returns the label image

core.getBarcodeImage() is an alias for getLabelImage

All of the above methods return PIL Image-objects. It actually took some discussion to figure out how to package the data. Since the SDK wraps around an HTTP-based API, we settled on representing pixels through Pillows. Pillows is the successor to the Python Image Library (PIL). The package should be installed for you automatically when you obtained Core java.

The following code shows all three representations of a slide side by side:

import java.awt.BorderLayout;
import java.awt.Image;
import javax.swing.ImageIcon;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;

import com.pathomation.*;

public class Test {

    public static void main(String[] args) {

        String slide = "C:/my_slides/alk_stain.mrxs";
        // displaying thumbnail image
        Image thumb = Core.getThumbnailImage(slide);
        displayImage(thumb);
        // displaying label image
        Image label = Core.getLabelImage(slide);
        displayImage(label);
        // displaying barcode image
        Image barcode = Core.getBarcodeImage(slide);
        displayImage(barcode);
    }
    
    public static void displayImage(Image image) {
        JFrame frame = new JFrame();
        JLabel lblImage = new JLabel(new ImageIcon(image));
        JPanel mainPanel = new JPanel(new BorderLayout());
        mainPanel.add(lblImage);
        frame.add(mainPanel);
        frame.setVisible(true);
    }    
}

The output is as follows:

Note that not all WSI files have label / barcode information in them. In order to determine what kind of associated images there are, you can inspect a SlideInfo dictionary first to see what’s available:

Map<String, Object> info = Core.getSlideInfo(slide);
System.out.print(Collections.singletonList((HashMap<String, Object>) info.get("AssociatedImageTypes")));

AssociatedImageTypes may refer to more than thumbnail or barcode images, depending on the underlying file format. The most common use of this function is to determine whether a barcode is included or not.

You could write your own function to determine whether your slide has a barcode image:

protected Boolean slideHasBarcode(String slide) {
    Map<String, Object> info = Core.getSlideInfo(slide);
    return ((HashMap<String, Object>) info.get("AssociatedImageTypes")).keySet().contains("barcode");
}

Whole Slide Images

If you already know about pyramidical image files, feel free to skip this paragraph. If you don’t, sticks around; it’s important to understand how microscopy data coming out of slide scanners is structured to be able to manipulate it.

It all starts with a physical slide: a physical slide is a thin piece of glass, with the dimensions

When a physical slide is registered in a digital fashion, it is translated into a 2-dimensional pixel matrix. At a 40X magnification, it takes a grid of4 x 4 pixels to represent 1 square micrometer. We can also say that the image has a resolution of 0.25 microns per pixel. This is also expressed as 4 pixels per micron (PPM).

All of this means that in order to present our 5 cm x 2 cm physical specimen from the first part of this tutorial series in a 40X resolution we need (5 * 10 * 1000 * 4) * (2 * 10 * 1000 * 4) = 200k x 80k = 16B pixels

Now clearly that image is way too big to load in memory all at once, and even with advanced compression techniques, the physical sizes of these is roughly around one gigabyte per slide. So what people have thought of is to package the image data as a pyramidal stack.

Pyramid of Cestius as a metaphore for pyramidal stack images. By Francesco Gasparetti from Senigallia, Italy – Piramide Cestia, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=2614848

Ok, perhaps not that kind of pyramid…

But you can imagine a very large image being downsampled a number of times until it receives a manageable size. You just keep dividing the number of pixels by two, and eventually you get a single image that still represents the whole slide, but is only maybe 782 x 312 pixels in size. This then becomes the top of your pyramid and we label it as zoomlevel 0.

At zoomlevel 1, we get a 1562 x 624 pixel image etc. It turns out that our original image of 200k x 80k pixels is found at zoomlevel 8. Projected onto our pyramid, we get something like this:

Worked out example (showing the different zoomlevels) of a pyramidal stack for a concrete whole slide image.

So the physical file representing the slide doesn’t just store the high-resolution image, it stored a pyramidal stack with as many zoomlevels as needed to reach the deepest level (or highest resolution). The idea is that depending on the magnification that you want to represent on the screen, you read data from a different zoomlevel.

Tiles

The pyramid stack works great up to certain resolution. But pretty quick we get into trouble and the images become too big once again to be shown in one pass. And of course, that is eventually what we want to do: Look at the images in their highest possible detail.

In order to work around this problem, the concept of tiles is introduced. The idea is that at each zoomlevel, a grid overlays the image data, arbitrarily breaking the image up in tiles. This leads to a representation like this:

Now, for any area of the slide that we want to display at any given time to the end-user, we can determine the optimal zoomlevel to select from, as well a select number of tiles that are sufficient to show the desired “field of view”, rather than asking the user to wait to download the entire (potentially huge!) image. This goes as follows:

Or, put the other way around (from the browser’s point of view):

So there you have it: whole slide images are nothing but tiled pyramid-shaped stacks of image data.

Tiles in PMA.start

We can access individual tiles within the tiled stack using PMA.start, but before we do that we should first look some more at a slide’s metadata.

We can start by making a table of all zoomlevels the tiles per zoomlevel, along with the magnification represented at each zoomlevel (using library TableSaw) :

import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;

import com.pathomation.*;
import tech.tablesaw;

public class Test {

    public static void main(String[] args) {

        List<Map<String, Object>> levelInfos = new ArrayList<Map<String, Object>>();
        String slide = "C:/my_slides/alk_stain.mrxs";
        List<Integer> levels = Core.getZoomLevelsList(slide);
        //Initialize the components for TableSaw
        DoubleColumn colResX = DoubleColumn.create("resX");
        DoubleColumn colResY = DoubleColumn.create("resY");
        DoubleColumn colTilesX = DoubleColumn.create("tilesX");
        DoubleColumn colTilesY = DoubleColumn.create("tilesY");
        DoubleColumn colApproxMag = DoubleColumn.create("approxMag");
        DoubleColumn colExactMag = DoubleColumn.create("exactMag");
        for (int lvl : levels) {
            int resX = Core.getPixelDimensions(slide, lvl).get(0);
            int resY = Core.getPixelDimensions(slide, lvl).get(1);
            List<Integer> tilesXyz = Core.getNumberOfTiles(slide, lvl);            
            colResX.append(resX);
            colResY.append(resY);
            colTilesX.append(tilesXyz.get(0));
            colTilesY.append(tilesXyz.get(1));
            colApproxMag.append(Core.getMagnification(slide, lvl, false));
            colExactMag.append(Core.getMagnification(slide, lvl, true));
        }
        System.out.print(slide);
        Table results = Table.create("Results", colResX, colResY, colTilesX, colTilesY, colApproxMag, colExactMag);
        results.print();
    }
}

The result for our alk_stain.mrxs slide looks as follows:

Now that we have an idea of the number of zoomlevels to expect and how many tiles there are at each zoomlevel, we can request an individual tile easily. Let’s say that we wanted to request the middle tile at the middle zoomlevel:

import java.awt.BorderLayout;
import java.awt.Image;
import java.util.ArrayList;
import javax.swing.ImageIcon;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;

import com.pathomation.*;

public class Test {

    public static void main(String[] args) {

        String slide = "C:/my_slides/alk_stain.mrxs";
        List<Integer> levels = Core.getZoomLevelsList(slide);
        int lvl = levels.get((int) (levels.size() / 2));
        List<Integer> tilesXyz = Core.getNumberOfTiles(slide, lvl);
        int x = (int) (tilesXyz.get(0) / 2);
        int y = (int) (tilesXyz.get(1) / 2);
        Image tile = Core.getTile(slide, x, y, lvl);
        displayImage(tile);
    }
    
    public static void displayImage(Image image) {
        JFrame frame = new JFrame();
        JLabel lblImage = new JLabel(new ImageIcon(image));
        JPanel mainPanel = new JPanel(new BorderLayout());
        mainPanel.add(lblImage);
        frame.add(mainPanel);
        frame.setVisible(true);
    }
}

This should pop up a single tile:

.Ok, perhaps not that impressive.

In practice, you’ll typically want to loop over all tiles in a particular zoomlevel. The following code will show all tiles at zoomlevel 1 (increase to max_zoomlevel at your own peril):

List<Integer> tileSz = Core.getNumberOfTiles(slide, 1); // zoomlevel 1
IntStream.range(0, tileSz.get(0)).forEach(xTile -> {
    IntStream.range(0, tileSz.get(1)).forEach(yTile -> {
        Image tile = Core.getTile(slide, xTile, yTile, 1);
        displayImage(tile);
    });
});

The advantage of this approach is that you have control over the direction in which tiles are processed. You can also process row by row and perhaps print a status update after each row is processed.

However, if all you care about is to process all rows left to right, top to bottom, you can opt for a more condensed approach:

for (Image tile : Core.getTiles(slide, tileSz.get(0), tileSz.get(1), 4) {
    //converting the image into N-Dimensional Array
}

The body of the for-loop now processes all tiles at zoomlevel 4 one by one and converts them into an array structure, ready for image processing to occur, e.g. through opencv.