– a new AI-based tool for microscopy
Sharp images need a lot of light or a long exposure time. But too much light damages cells, which are also always in motion. In microscopy, images are therefore often taken at short exposure times and low laser power and are blurred and noisy. Computer scientists at the Center for Systems Biology/Max-Planck institute for cell biology and genetics in Dresden developed software that avoids this problem. The artificial intelligence-based software CARE, Content-aware image restoration, can calculate razor-sharp microscopy data from noisy images. This enables biologists to obtain high-resolution images and films without exposing the cells to the risk of light toxicity.
Big Data Problems in microscopy
The researchers involved had been focusing on Big Data in microscopy for some. In vivo and light sheet microscopy quickly amasses giga or terabytes of data. These data need de-noising and de-convolution before analysis, a time-consuming and computationally intensive step. Recently, computer science have more and more used machine learning with neural networks to solve complex tasks. In machine learning computers are fed data and learn to solve a specific problem. This approach is being the recent successes of computers in Chess and Go, and also powers the translation tool Deepl. It was thus not far fetched, to also use machine learning in bioimage data de convolution.
Image de-convolution thanks to AI
The first author of the study, Martin Weigert, says: “It was already known that machine learning for 2D images actually delivers very good results. What we had to do was transfer this to biological data.” Martin tried it out just before heading home for Christmas in 2016. The results were so overwhelming that he initially assumed he had made a mistake. Martin Weigert: “I called Loïc [Royer, another author] over and we discussed how this is impossibly good, this can’t be right, there must be some mistake!
It wasn’t a mistake. Martin Weigert was finally convinced when he saw pictures of fruit flies. In the noisy original image, no structures were visible to the eye. But after a machine learning network was trained, it calculated immaculate images showing detailed distributions of protein in the membrane of fruit fly wings.
To really adapt machine learning for biological images, the research team had to overcome several challenges. Microscopic images are often three- or four-dimensional, biological materials have different densities, microscopes have certain point-spread-functions and cameras have certain sensitivities. But eventually, CARE was born.
Happy embryos thanks to AI
The new machine learning tool CARE not only accelerates de-convolution but also has other advantages. Besides large data sets, photo-toxicity is an ever-present problem of in vivo microscopy. When embryos are imaged for days, die while being observed.
Akanksha Jain is interested in the early development of Tribolium beetle and says, “Even if my beetles don’t die, I am concerned that the laser power I need for good images will cause artefacts.” For Akanksha, postdoc in Pavel Tomancak’s laboratory, this problem is solved with CARE: “Once the network is trained, I can image at only 0.1% laser power, you literally see nothing, and then reconstruct the images with CARE. It’s mind-blowing how good that works.” So far, CARE has been able to convince in every tissue and model organism tested. Martin Weigert and his colleagues have already used it with mouse liver, zebrafish retina, fruit fly embryos, flatworms, Drosophila wings and Akanksha’s beetles.
How does CARE work?
To train a CARE network you need only a few pictures. These images are specific for a microscope set-up, tissue and fluorescent markers and image-pairs must be acquired at low and high laser intensity. CARE then calculate the correction values from the typical deviations.
CARE in application
In practice, the training set of images could be taken at the beginning and the end of an experiment, or also much later. And, once trained, a CARE network can be re-used indefinitely for future (and in fact also for past) experiments.
The demands on the microscope are also low: CARE can be used for any microscope type. The only requirement is that users may vary the laser intensity for the training images. Akanksha Jain confirms this: “CARE does not change the type of experiments I do, but it increases the possibilities of how I can capture images: it becomes faster and more robust”.
Florian Jug also sees the use of CARE for biologists as unproblematic. In his opinion, the biggest difficulty is “that people can’t install it on a computer.” For this, the team has taken precautions: the publication is accompanied by a detailed online documentation explaining step by step how CARE works with Windows or Linux.
Careful with CARE?
An important question is always whether software can introduce artefacts in the data. After Martin Weigert got CARE up and running in just a few days, it took the team about a year to be fully convinced that it works error-free. “We had to find out: can I trust the network” says Florian Jug. The team first showed that independently trained CARE networks deliver comparable results. Similar to two people solving the same math problem but using a different path. The team also controlled that the variance isn’t visible on a single pixel basis, demonstrating that CARE does not insert or changes any data, but merely sharpens what is present already.
CARE thus allows biologist to get better images without having to invest in better microscopes. Martin Weigert is finishing his doctoral thesis, but the next innovations are already in the pipeline. A method to calculate optimized images without reference images is in the planning stage.
This is a English version of an article that also appeared in 3/2019 of Laborjournal.