Japanese researchers take a reinforcement learning approach to study the process of fluid mixing during laminar flow


Fluid mixing is a critical component in many industrial and chemical processes. Pharmaceutical mixing and chemical reactions, for example, may require a homogeneous fluid mixture. Achieving this mixture faster and with less energy would significantly reduce the associated costs. In reality, however, most mixing processes are not mathematically optimized and instead rely on empirical methods based on trial and error. Turbulent mixing, which uses turbulence to mix fluids, is one option but is problematic because it is difficult to maintain (as in micro-mixers) or damages the mixed materials (as in bioreactors and food mixers).

Can we instead obtain a mixture optimized for laminar flows? To answer this question, a team of researchers from Japan, in a new study, turned to machine learning. In their study published in Scientific Reports, the team used an approach called “reinforcement learning” (RL), in which intelligent agents act in an environment to maximize cumulative reward (as opposed to instantaneous reward).

“Since RL maximizes the cumulative reward, which is time-global, one might expect it to be suitable for solving the problem of efficient fluid mixing, which is also a time-global optimization problem. “, explains associate professor Masanobu Inubushi, the corresponding author of the study. “I personally believe that finding the right algorithm for the right problem is important rather than blindly applying a machine learning algorithm. Fortunately, in this study, we were able to bridge the two areas ( fluid mixing and reinforcement learning) after considering their physical and mathematical characteristics.” The work included contributions from Mr. Mikito Konishi, a graduate student, and Professor Susumu Goto, both from Osaka University.

A major obstacle awaited the team, however. Although RL is suitable for global optimization problems, it is not particularly well suited for systems involving high-dimensional state spaces, i.e. systems requiring a large number of variables for their description . Unfortunately, the fluid mixture was just such a system.

To solve this problem, the team adopted an approach used in the formulation of another optimization problem, which allowed them to reduce the dimension of the state space for fluid flow to one. Simply put, fluid motion could now be described using a single parameter!

The RL algorithm is usually formulated in terms of the “Markov decision process” (MDP), a mathematical framework for decision making in situations where the outcomes are partly random and partly controlled by the decision maker. Using this approach, the team showed that RL was effective in optimizing fluid mixing.

“We tested our RL-based algorithm for the two-dimensional fluid mixing problem and found that the algorithm identified efficient flow control, which resulted in exponentially fast mixing without any prior knowledge,” says Dr. Inubushi. “The mechanism underlying this efficient mixing has been explained by looking at the flux around fixed points from the perspective of dynamical systems theory.”

Another significant advantage of the RL method was effective transfer learning (applying acquired knowledge to a different but related problem) of the trained “mixer”. In the context of fluid mixing, this implied that a mixer driven at a certain Péclet number (the ratio of rate of advection to rate of diffusion in the mixing process) could be used to solve a mixing problem at a another Peclet number. This greatly reduced the time and cost of training the RL algorithm.

While these results are encouraging, Dr. Inubishi stresses that this is still the first step. “There are still many problems to solve, such as applying the method to more realistic fluid mixing problems and improving RL algorithms and their implementation methods,” he notes.

While it is certainly true that two-dimensional fluid mixing is not representative of actual mixing problems in the real world, this study provides a useful starting point. Additionally, although it focuses on mixing in laminar flows, the process is also scalable to turbulent mixing. It is therefore versatile and has potential for major applications in various industries using fluid mixing.


Reference DOI: https://doi.org/10.1038/s41598-022-18037-7

About Tokyo University of Science Tokyo University of Science (TUS) is a well-known and respected university, and the largest private research university specializing in science in Japan, with four campuses in central Tokyo and its suburbs and in Hokkaido. Founded in 1881, the university has continuously contributed to the scientific development of Japan by instilling a love of science in researchers, technicians and educators.

With a mission to “create science and technology for the harmonious development of nature, human beings and society”, TUS has undertaken a wide range of research from basic science to applied science. TUS has taken a multidisciplinary approach to research and undertaken intensive studies in some of today’s most vital areas. TUS is a meritocracy where the best in science is recognized and nurtured. It is the only private university in Japan that has produced a Nobel laureate and the only private university in Asia to produce Nobel laureates in the field of natural sciences.

Website: https://www.tus.ac.jp/en/mediarelations/

About Associate Professor Masanobu Inubushi of Tokyo University of Science Masanobu Inubushi is currently an associate professor at Tokyo University of Science, Japan. He received his undergraduate degree in 2008 from Tokyo Institute of Technology, Japan. He then obtained his PhD in Mathematics from the Research Institute of Mathematical Sciences (RIMS) of the Graduate School of Kyoto University in 2013. After working at NTT, Communication Science Laboratories from 2013 to 2018, he joined Osaka University as an Assistant Professor in 2018. Dr. Inubushi has over 25 published research papers which have been cited over 400 times. His research interests include fluid mechanics, chaos theory, mathematical physics and machine learning.

Funding Information This work was partially supported by JSPS Grant-in-Aid for Early-Career Scientists No. 19K14591and JSPS Grants-in-Aid for Scientific Research Nos. 19KK0067, 20H02068, 20K20973 and 22K03420.


Comments are closed.