Sergei Kalinin: Doing it differently
Image: From AFM to STEM Sergei Kalinin, ORNL, is pushing back the boundaries of microscopy.
Dr Sergei Kalinin is doing something very different with electron microscopy.
While most microscopists will painstakingly strive to preserve the form of the material being imaged, his team at Oak Ridge National Laboratory uses the electron beam to nudge atoms out of alignment and and now wants to build structures.
Kalinin and colleagues have already moved atoms into small clusters and by using artificial intelligence to control the electron beam precisely, they hope to swiftly and automatically assemble complex structures, atom by atom, (see 'Meet the team').
Such a system could fabricate next-generation devices for quantum computing and spin sensing, and Kalinin can’t wait.
STEM image of silicon cluster within graphene [Ondrej Dyck/ORNL]
“For decades, Eigler’s [scanning tunnelling microscopy] method was the only technology to manipulate atoms one by one, but we’ve demonstrated a second approach with an electron beam in the scanning transmission electron microscope,” points out Kalinin.
“Right now we have a team of researchers with vastly different backgrounds which allows us to do things that are so very exciting,” he adds. “We really are boldly going where no-one has gone before.”
Growing up in Moscow before the collapse of the Soviet Union, the young Kalinin excelled in Chemistry.
During High School he took advanced placement classes and competed in the International Chemistry Olympiads where his theory work was ranked world second, exempting him from high-school exams and guaranteeing his admission into any University in Russia.
As he puts it: “This was a really nice effect of being part of these Olympiads.”
On leaving school – already with a published paper under his belt – Kalinin headed for Moscow State University to study Materials Science, which in his words, ‘sounded cool’.
At the time, the Soviet Union was investing heavily in this field, and his chosen university had a rigorous program of chemistry, physics and mathematics while providing the opportunity for its students to conduct original research.
“Classes would run from 9am to 5:30pm and then you still had to find the time to do research and homework,” says Kalinin. “To a certain degree we were escaping from our surrounding reality, but we were all so curious about doing the research and the role of our supervisors was to keep this sense of curiosity safe.”
The Soviet Union dissolved during Kalinin’s last year in high school, and while many of his peers eventually left science, he steadfastly remained.
I was naive and thought that it would be enough to see the atoms, but once you see them, you really want to understand what they are doing. Sergei Kalinin.
During his studies he spent six months at the Pohang University of Science and Technology – POSTECH – South Korea, where he made his first all-important foray into the world of microscopy.
The group he joined was set to work with atomic force microscopy but when he reached the university, the laboratory hadn’t yet been set up.
Undeterred the young Kalinin spent his time digesting scientific literature and on returning to Moscow was fully-versed in the theory behind AFM as well as the physics and mechanics of nanoscale systems.
“I just read all the available literature at the time and once back in Moscow, a colleague recommended me to the laboratory of Dawn Bonnell at the University of Pennsylvania, for my PhD,” he says. “Dawn was working with scanning probe microscopy so I impressed her with my book knowledge and she accepted me as her student.”
To direct-write the logo of the Department of Energy’s Oak Ridge National Laboratory, Kalinin and colleagues started with a gray-scale image, then used the electron beam of their AC-STEM to induce palladium from a solution to deposit as nanocrystals. [ORNL]
Leaving Russia for US in 1998, as Kalinin says, ‘wasn’t particularly easy, but the obvious, decision’. The nation was undergoing extreme social change and its economy had crashed.
“My colleagues and I had done our best to stay within the scientific world and survive these changes so compared to this, the move to the US was easy,” he says. “[Americans] were really welcoming and when all was said and done, I was very young and just wanted to do science.”
So Kalinin joined the Bonnell group and immediately started to investigate nanoscale electric phenomena in oxide materials using scanning probe microscopes. According to the researcher, the field was very small at the time and his remit was broad.
During this time he modified the available microscopes and used myriad methods, including variable temperature AFM, Kelvin Probe Force microscopy and Piezoresponse Force Microscopy, to characterise the surfaces of ferroelectric ceramics such as BaTiO3 and SrTiO3, as well as polycrystalline BiFeO3 and ZnO ceramics and more.
Part of his work involved the use of a biased scanning probe microscopy tip to write ferroelectric domains, and then use chemical reactivity of domains to create metal patterns.
This fascination with modifying imaging tools to modify and create structures would persist for the next twenty years.
For nearly two decades, Kalinin (pictured here with ORNL colleague, Rama K Vasudevan) has been developing microscopy methods. [ORNL]
Kalinin published prolifically during this time adding a mighty thirty publications to bring his peer-reviewed publication count to 55 before even getting his doctorate.
As he says: “I had discovered that in the scientific world, if 'it ain’t published then it ain’t done' and of course, having a good set of publications gives you credibility and is very helpful when exploring future career opportunities.”
Still, Kalinin wasn’t destined to spend very long at Pennsylvania. His doctorate had thrown up many questions over oxide ceramic behaviour and his results had been strongly influenced by imaging in ambient conditions.
So come 2002, he accepted a Wigner fellowship at Oak Ridge National Laboratories to continue imaging oxides but this time under ultrahigh vacuum and at a higher resolution.
“Once you get to the higher resolution measurements that imaging in an ultrahigh vacuum allows, you take a fundamentally new step,” points our Kalinin. “You are no longer studying solids on the single nanometre scale in the presence of water and unknown contaminates, but instead you are studying at the atom-scale.”
“At the time I was naive and thought that it would be enough to just see the atoms. However, once you see them, you really want to understand what they are doing, and that’s even more difficult,” he adds.
Using piezoresponse force microscopy, Kalinin and colleagues were the first to detect ferroelectric domains (seen as red stripes) in the simplest known amino acid, glycine, in 2012. [ORNL]
In his early days a ORNL, Kalinin pursued his interest in characterising oxide surfaces, using SPM and was also determined to image very clean surfaces at atomic resolution.
He spent many years with SPM group leader, Art Baddorf, developing a massive system for pulsed laser deposition to synthesize advanced materials, and image and characterise samples in-situ.
“I designed a lot of this system and many elements are still operational today,” he says. “It was a really great experience but I realised whilst working on this that scanning probe microscopy is very time consuming, and understanding what the results mean is even worse.”
What’s more, Kalinin was also grappling with his oxide observations.
“It turned out that when we grew many of our materials, the surface actually looked like a moonscape,” he says. “At atomic resolution you see that [the surface] is highly disordered, so you ask yourself, what does this mean and what physics or chemistry can we extract from observations?”
Crucially for Kalinin, and his quest to understand atoms, his new laboratory was next door to the group of Professor Steve Pennycook, who himself was intent on stretching high resolution STEM imaging to its limits.
Sergei Kalinin, Stephen Pennycook and ORNL colleagues used STEM to sculpt 3D nanoscale features in a complex oxide material. [ORNL]
Working alongside the world-class electron microscopist and his team, Kalinin quickly spotted, as he says, ‘tremendous potential for synergy’ between the fields of scanning probe microscopy and electron microscopy.
So with this in mind, he started to seriously consider what could be learned from the atomically resolved images of materials obtained from the two microscopies. Kalinin’s interest in scanning probe microscopy has never waned but at this point in time, he was quick to realise the advantages of electron microscopy.
As he highlights, electron microscopy could deliver many atomically-resolved images every day.
“But in contrast, with scanning probe microscopy, an image could take a month to obtain, which would then represent some complex function of the electronic structure,” he adds. “So I soon realised that if you want to understand your materials on the atomic level, then you should start with electron microscopy.”
With this in mind, Kalinin started to use neural networks to extract more and more information from mesoscopic and atomically-resolved images.
Compared to scanning probe microscopy, electron microscopy has tremendous potential for growth as there are more electron microscopes that can do this more simply. Sergei Kalinin.
Come 2009, Kalinin, Pennycook and colleagues were successfully using linear neural network algorithms with STEM-electron energy loss spectroscopy to map the composition across oxide interfaces.
Several papers exploring applications of machine learning in scanning probe microscopy quickly followed, with the researchers publishing ‘Towards the thinking microscope’ in Microscopy and Microanalysis.
The article laid out their intention to use grid search and data mining algorithms to match 2D image data to 3D EELS data, to directly recognise structures and fine details of electronic properties on an atomic level.
In the intervening years, Kalinin has swiftly built on his initial idea of using machine learning and neural networks to extract detailed information from atomically-resolved images with a view to developing advanced materials for energy and IT applications.
In recent research, he has used machine learning to identify material phase transitions in nanometre-scale volumes using atomic force microscope data.
What’s more, he has also used convolutional neural networks to characterise molecular assembly on metallic surfaces in scanning tunnelling microscopy experiments. And he has developed data-driven manifold leaning approaches to analyse 4D-STEM datasets.
But Kalinin’s research interests extend beyond making the most of atomically-resolved images.
When working with Pennycook’s group, he noticed how very motivated his fellow researchers were to avoid atom-displacement damage in their samples. However, he quickly realised that this beam damage phenomenon could be harnessed to control and move atoms to specific locations.
“Steve Pennycook and his researchers were showing datasets of atoms jumping in the crystal lattice while under the electron beam and I made the connection instantly,” he says. “In scanning probe microscopy, we were used to seeing the atoms move under the effect of the tip so I asked myself, you can make atoms move where you want here, so why not do the same with electron microscopy?”
It was 2014, and Kalinin and Jesse instantly got to work investigating this prospect further, which he knew wasn’t going to be easy.
As Kalinin puts it: “The problem with atomic manipulation via electron microscopy is you have to put together a lot of very dissimilar parts, such as instrumental control, sample preparation, sample doping and it’s not so easy as all of this is outside of the normal electron microscopy workflow.”
In 2014 and using STM, Kalinin and ORNL colleagues delivered “distortion maps” (right) that brought into view domains within a perovskite manganite that were not easily identified in the raw images (left). They generated images of the manganite surface down to the level of 30 picometres. [ORNL]
Still, he and colleagues were quick to use STEM to sculpt crystalline oxides at the atomic-level and come 2017 they had used a sub-atomically focused STEM beam to manipulate single atoms in graphene.
More recently, the researchers have used Fourier transforms to implement active feedback control and automatically control electron beam motion and, building on their past artificial intelligence advances, are combining this with deep learning to automate atom manipulation in semiconductors and more.
And in a breakthrough that rocked the world of microscopy just last year, they introduced silicon substitutional defects into graphene and went onto manipulate the defects to form dimers, trimers and more complex structures.
Importantly, along the way, this effort has nucleated a team of researchers spanning many disciplines. For example, with a background in electrical engineering and instrumental control, Stephen Jesse is adept at building new controls and feedbacks, and modifying commercial tools with custom electronics.
Meanwhile electron microscopist, Ondrej Dyck, focuses on developing STEM for atom-by-atom fabrication.
As Kalinin points out: “With Ondrej, we needed someone that didn’t mind dedicating a lot of time to doing something that a normal electron microscopist would consider to be beam damage. He and Stephen Jesse make the “impossible” happen.”
Andrew Lupini from ORNL using the Nion UltraSTEM 200. [ORNL]
Team members, Andrew Lupini and Mark Oxley specialise in electron microscopy optics and control systems, with Lupini having worked alongside Nion’s Ondrej Krivanek during the development of aberration correction. And Rama Vasudevan and Maxim Ziatdinov have made the huge leap from Physics into artificial intelligence.
So where next for Sergei Kalinin and his colleagues? Right now Kalinin is keen to convey how useful atom-by-atom fabrication via STEM is. As he highlights, he and colleagues have already demonstrated that electron microscopy atomic manipulation holds vast potential to atomic engineer quantum computing devices and sensors.
“Compared to scanning probe microscopy, electron microscopy also has tremendous potential for growth here as there are many more electron microscopes that can do this so much more simply,” he adds. “And there are many electron microscope manufacturers, so if there is a market, these companies can scale production very quickly.”
At the same time, Kalinin is eager to points out how very quick progress has been to date.
“Three years ago the simplest task such as extracting all the atoms in an image was a semi-manual task but we automated this with artificial intelligence a year ago,” he says.
“With these developments we are bringing together fields such as statistical physics and condensed matter physics as well as using tools that came from observational biology and astronomy,” he adds. “We are building tiny bridges between many different disciplines that all of a sudden are going to grow, this really is a very exciting time.”
Meet the Team
With Stephen Jesse, Ondrej Dyck, and Andy Lupini, Sergei Kalinin is pushing back the frontiers of electron microscopy, using its electron beam to re-arrange atoms, understand the mechanisms of the beam induced reactions via machine learning and explore physics of materials one atom at at time.
Together, they are harnessing electron microscopy to build structures. This work is supported by a US Department of Energy grant – Jesse is the principal investigator.
Artificial intelligence has been crucial to progress, and with Maxim Ziatdinov, Lukas Vlcek, and Rama Vasudevan, the researchers intend to use machine learning to study and predict material evolution, and create deep learning tools to direct electron beam induced transformations.