Extending PMA.core functionality with external scripts

Pathomation offers a comprehensive platform with a variety of technical components that help you build tailor-made digital pathology environments and set up custom workflows. Centered around PMA.core, our powerful tile server, everything else can be connected and integrated. You want a feature that PMA.core does not support or you have a script/code to implement it. Great! You can.

PMA.core allows the registration and execution of third party command line scripts via its External Scripts admin page and its Scripts Run API. You can utilize this to create anything you need from simple workflow tasks like batch renaming/moving of slides to fancy AI and tissue recognition algorithms. Assuming you have found one such a fancy script and you want to integrate it into PMA.core let’s see the process step by step.

Preparing for integration

As an example we will use the sample script (provided here) that recognizes tissue automatically using the OpenCV computer vision library and imports the recognized areas as annotations all in one go. The sample tissue recognition script requires Python and OpenCV to be installed and configured in your system. That script also requires some parameters to be executed successfully like the PMA.core server url, username and password and the path to the slide to analyze.
For this reason we will create an intermediate .bat file to facilitate the execution of the python script with the correct parameter values as described bellow(replace \path\to\script with the path the script is located).

python.exe "\PATH\TO\SCRIPT\AutoAnnotator.lite.py" -l %2 -u %3 -p %4 -t %5 -f %1

Registering a script

To register your script with PMA.core navigate to the Settings -> External Scripts page and click Add.

Adding a new external script with PMA.core interface

You need to provide the following information describing our script and the parameters required to execute it.

  • Name: A unique name for the script used to fetch it and distinguish from other, for our example enter Automatic tissue annotations
  • Command: The command line to execute, a bat file, cmd file or exe, for our example enter \path\to\script\AutoAnnotator.bat
  • Arguments: The arguments passed to the script, for our example enter {slidePath} {serverUrl} {username} {password} {targetDirectory}
  • Parameters: A dynamic list of parameters passed to the script by PMA.core, for our example enter the list of parameter as shown in the following image.
The settings required to register our script

In the arguments section any text you enter will be passed to the executed command as is. An exception to this is the text enclosed by curly brackets (for example {slidePath}). If the text inside the curly brackets equals a parameter name, PMA.core will replace it with the value supplied at the execution step. PMA.core supports three types of parameters that validates accordingly at execution phase: String, Number, Path. Number parameters are validated as a floating point value, and Path is validated by checking the existence and permissions of the specified value.
After clicking Save you should see your newly created script and its settings in the main index page

List of external scripts registered to PMA.core

Executing the script

Now that we have registered the script correctly we can execute it using the Scripts Run API. In the index page of the interface on the url column you can copy/paste a helper url with all the required parameters to execute the script. In our example the helper url is:

/scripts/Run?name=Automatic tissue annotations&slidePath={Path}&serverUrl={String}&username={String}&password={String}&targetDirectory={String}

and we will replace {Path} in slidePath with the virtual path to the slide we want to annotate, the {String} in serverUrl with the PMA.core serverUrl, the {String} in username with our PMA.core username, the {String} in password with our PMA.core password, and the {String} in targetDirectory with the path to a local temporary folder.

After executing the script using the API you will get a JSON response with the following fields:

{
  "ScriptName": The executed script name,
  "Success": A boolean value indicating whether the script executed successfully or not,
  "ErrorMessage": An optional error message if any occured,
  "Result": The output of the script 
}

After a successful execution of our script in a sample slide you should be able to see the generated annotations containing the tissues as recognized by the script’s algorithm.

The final result of the tissue recognition algorithm

Required files

Automatic tissue recognition python script (AutoAnnotator.lite.py)

PMA.UI on React.JS with Collaboration using PMA.live

PMA.UI is a Javascript library that provides UI and programmatic components to interact with Pathomation ecosystem. It’s extended interoperability allows it to display WSI slides from PMA.core, PMA.start or Pathomation’s cloud service My Pathomation. PMA.UI can be used in any web app that uses Javascript frameworks (Angular.JS, React.JS etc) or plain HTML webpages.
PMA.live (now called PMA.collaboration) is a Javascript library that provided functionality to share your PMA.UI app/page with other users keeping all parts of the page synchronized, allowing for drawing annotations simultaneously, share multiple viewports just like with multi-head microscopes but over the internet.
We will create an example React application and integrate both PMA.UI and PMA.live as an example. You can use an existing React application or create a new one using create-react-app e.x.

npx create-react-app CollaborationReact

First we have to install PMA.UI and PMA.live library using npm by running

npm i @pathomation/pma.ui
npm i @pathomation/pma.collaboration

inside application’s directory.

Next step is to add JQuery in our app. Open index.html file inside public directory and add

<script src="https://code.jquery.com/jquery-3.6.0.min.js"
    integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script>

in the head tag.

It’s time to implement the main functionality of PMA.UI and fire up 4 simultaneous slide viewports in the same page. We will use the PMA.UI components to easily do that namely the context, slideloader and autologin. So go ahead and replace the default App.js with the following initialization page

const imageSet = ["Reference/Aperio/CMU-1.svs", "Reference/Aperio/CMU-2.svs", "Reference/3DHistech/CMU-1.mrxs", "Reference/3DHistech/CMU-2.mrxs"];

const createSlideLoader = (context, element) => {
  return new SlideLoader(context, {
    element: element,
    overview: { collapsed: false },
    dimensions: { collapsed: false },
    scaleLine: true,
    annotations: { visible: true, labels: true, showMeasurements: false },
    digitalZoomLevels: 2,
    loadingBar: true,
    highQuality: true,
    filename: true,
    barcode: false
  });
}

function App() {
  const viewerContainer = useRef(null);
  const slideLoaderRefs = useRef([]);
  const context = useRef(new UIContext({ caller: "Vas APP" }));
  const [slideLoaders, setSlideLoaders] = useState([]);
  const location = useLocation();

  useEffect(() => {
    if (slideLoaderRefs.current.length == 0) {
      slideLoaderRefs.current = [...Array(20)].map((r, i) => slideLoaderRefs[i] || createRef());
    }
    let autoLogin = new AutoLogin(context.current, [{ serverUrl: pmaCoreUrl, username: username, password: password }]);

  }, []);

  useEffect(() => {
    if (slideLoaderRefs.current.filter(c => !c || !c.current).length > 0) {
      return;
    }

    let slLoaders = [];
    for (let i = 0; i < slideLoaderRefs.current.length; i++) {
      slLoaders.push(createSlideLoader(context.current, slideLoaderRefs.current[i].current));
    }

    setSlideLoaders(slLoaders);
  }, [slideLoaderRefs.current]);

  return (
    <div className="App">
      <div ref={viewerContainer} className="flex-container">
        {slideLoaderRefs.current && slideLoaderRefs.current.map((r, i) =>
          <div className={"flex-item"} key={i} ref={slideLoaderRefs.current[i]}></div>)
        }
      </div>
    </div >
  );
}

export default App;

To properly show all 4 viewers in the same page we need some css to style it up, so we need to add this to index.css. This will split the page to a grid of 2×2 viewers using css flex.

.flex-container {
  display: flex;
  flex-direction: row;
  flex-wrap: wrap;
  width: 100%;
  height: 850px;
}

.flex-item.pma-ui-viewport-container {
  flex: 0 0 50%;
  height: 400px
}

.ml-1 {
  margin-left: 15px

Well that was easy to set up!

Collaboration

So let’s synchronize this page for all the user’s joining so they can see and interact with the same slides. For this we will be using the PMA.live server and to pma.collaboration package we installed earlier. To enable users to collaborate they have to join the same session, as it is called, one user will be the master of the session which controls the current viewports, slides and annotations(even though this can be changed with a setting for all users with a setting called EveryoneInControl).

PMA.live uses SignalR and the WebSocket protocol to achieve real time communication between participants, so we need to include this scripts in our page. We can include this scripts in the index.html as we did for jQuery, but we need to be sure that the scripts are properly loaded before trying to initialize any PMA.collaboration in our React application. So we will use the same trick used by Google Maps to load the scripts asynchronously and notify our React app when they are ready. So create a new file called collaborationHelpers.js with the following function.

export const loadSignalRHubs = (collaborationUrl, scriptId, callback) => {
    const existingScript = document.getElementById(scriptId);
    if (!existingScript) {
        const script = document.createElement('script');
        script.src = `${collaborationUrl}bundles/signalr`;
        script.id = scriptId;
        document.body.appendChild(script);
        script.onload = () => {
            const script2 = document.createElement('script');
            script2.src = `${collaborationUrl}signalr/hubs`;
            script2.id = scriptId + "hubs";
            document.body.appendChild(script2);

            script2.onload = () => {
                if (callback) {
                    callback();
                }
            };
        };
        return;
    }
    
    if (existingScript && callback) {
        callback()
    };
};


To notify our React app that the scripts are ready and to proceed with the initialization we need to create a new state in the App.js page called loadedScripts which we will set to true when the scripts are loaded in our previous useEffect function
useEffect(() => {
    if (slideLoaderRefs.current.length == 0) {
      slideLoaderRefs.current = [...Array(20)].map((r, i) => slideLoaderRefs[i] || createRef());
    }

        let autoLogin = new AutoLogin(context.current, [{ serverUrl: pmaCoreUrl, username: "zuidemo", password: "zuidemo" }]);
    loadSignalRHubs(collaborationUrl, "collaboration", () => {
      setLoadedScripts(true);
    });
  }, []);

So now everything is ready to establish a connection to the PMA.live backend, and also joining a session (joining a non-existing session will just create a new one) 
const initCollaboration = (nickname, isMaster, getSlideLoader, collaborationDataChanged, chatCallback) => {
  return Collaboration.initialize(
    {
      pmaCoreUrl: pmaCoreUrl,
      apiUrl: collaborationUrl + "api/",
      hubUrl: collaborationUrl + "signalr",
      master: isMaster,
      getSlideLoader: getSlideLoader,
      dataChanged: collaborationDataChanged,
      owner: "Demo",
      pointerImageSrc: "…",
      masterPointerImageSrc: "…"
    }, []).
    then(function () {
      var sessionName = "DemoSession";
      var sessionActive = false;
      var everyoneInControl = false;
      return Collaboration.joinSession(sessionName, sessionActive, nickname, everyoneInControl);
    }).
    then(function (session) {
      // after join session
      ////var appData = Collaboration.getApplicationData();
      if (isMaster) {
        Collaboration.setApplicationData({ a: 1 });
      }
    });
}

The initialize method tells to the PMA.live Collaboration static object where to find PMA.core, PMA.live backend, whether or not the current user is the session owner, what icons to show for the users’ and the master’s cursor and accepts a couple of callback functions. The joinSession method will create a session if it does not exist and then join it. If the session doesn’t exist, it is possible to specify whether or not it is currently active and if all users can take control of the synced viewports or only if the session owner can do this. Once the session has been created, only the session owner can modify it and change it’s active status or give control to others.

In order for PMA.live to be able to sync slides, it has to know the viewports it should work with. Earlier in our code we created an array of maximum of 20 slideloaders which we kept in a react ref object. Now let’s go back to the implementation of the “getSlideLoader” callback that we used during the initialization of PMA.live. This function will be called by PMA.live when it needs to attach to a viewport in order to control it. So we will need to return the appropriate slideLoader from this React Ref array with this function

const getSlideLoaderCb = (index, totalNumberOfImages) => {
    if (!master && totalNumberOfImages < numberOfImages) {
      for (let i = totalNumberOfImages; i < numberOfImages; i++) {
        slideLoaders[i].load(null);
      }
      setNumberOfImages(totalNumberOfImages);
    }

    return slideLoaders[index];
  }

So now we can initialize the collaboration in a useEffect which executes after the SignalR and Hubs scripts are properly initialized

useEffect(() => {
    if (loadedScripts && viewerContainer.current && slideLoaderRefs.current && slideLoaders.length > 0) {
      if (slideLoaderRefs.current.filter(c => !c || !c.current).length > 0) {
        return;
      }

      if (collaborationInit) {
        return;
      }

      initCollaboration("demo user", master, (index, totalNumberOfImages) => {
        return getSlideLoaderCb(index, totalNumberOfImages);
      },
        () => {
          let data = Collaboration.getApplicationData();
          let session = Collaboration.getCurrentSession();
          setCollaborationData({ data: data, session: session });
        })
        .then(() => {
          setCollaborationInit(true);
        });
    }
  }, [loadedScripts, viewerContainer, master, slideLoaders.length, collaborationInit, slideLoaderRefs, slideLoaderRefs.current.length]);

Finally, let’s talk about the “collaborationDataChanged” callback. Besides the out of the box slide syncing capabilities, it gives you the ability to exchange data between the users, in real time. This could be useful for example if you wanted to implement a chat system on top of the session. Every user can change the session’s data by invoking the Collaboration.setApplicationData method. It accepts a single object that will be shared among users. Whenever this method is called, all other users will receive it through the collaborationDataChanged callback. To do this is a React way we simply set the application data to a React state object whenever the callback is called.

To allow other users to join our application as guests we will implement a query string parameter called master. When this parameter is set to false users joining this session will be guests. We keep this value in a React state called master. So we change our initial useEffect function to add this

var urlSP = new URLSearchParams(location.search);
if (urlSP.get("master") === "false") {
   setMaster(false);
}

Congratulations you’ve done it! You can now have a working React application with PMA.UI and collaboration featured enabled.


You can download a complete demo here

Additional developer resources are provided here