In this study, a short dynamic visualization titled “Dancing with Atoms” was produced in honor of the late Irish mathematical physicist Sheila Tinney (1918–2010), who was the first Irish-born and -raised woman to receive a doctorate in the mathematical sciences. The visualization was inspired by Tinney’s ground-breaking work on crystal lattice vibrations and consisted of an animation showing an atomic lattice structure vibrating in three-dimensions based on data derived from a musical piece performed by her son, award-winning pianist Hugh Tinney. The acoustic signal processing and visualization were conducted in the new Science Foundation Ireland-funded “Tinney” high-performance computing cluster in Trinity College Dublin, Ireland. Potential applications of this type of STEAM (Science, Technology, Engineering, Arts, and Mathematics) interdisciplinary work include its use as a novel visual aid to hearing tests or as a tool for training musicians.
Research Article
“Dancing with Atoms”: A tribute to Sheila Tinney
https://doi.org/10.21203/rs.3.rs-2467748/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
animation
computational chemistry
music
sound
STEAM
According to UNESCO’s Institute for Statistics, less than 30% of the world's researchers are women [1]. In recent years, governments and funders have been encouraging and supporting active involvement of women in STEM (Science, Technology, Engineering, and Mathematics). As part of that strategy, we recognize and celebrate pioneering women who had distinguished careers in STEM. One such woman was Irish mathematical physicist Sheila Tinney (1918–2010), the first Irish-born and -raised woman to receive a doctorate in the mathematical sciences [2]. Her 1941 PhD from the University of Edinburgh on the stability of crystal lattices [3] was completed in just two years under the supervision of Nobel laureate (1954, Physics) Max Born. Sheila greatly contributed across a wide range of scientific topics including crystal lattices, wave mechanics, quantum electrodynamics, cosmic radiation, and meson theory [4–8]. Throughout her career, she collaborated with top scientists such as Erwin Schrödinger, Hideki Yukawa (both Nobel laureates) and Walter Heitler, at a time when STEM fields were almost exclusively male (see Fig. 1).
STEAM refers to the close association of the Arts to STEM disciplines. Music was very important to Sheila, and she was a gifted and dedicated amateur musician, an interest passed on to her from her mother. She in turn passed this love of music on to her son Hugh Tinney, who went on to become an award-winning concert pianist and Professor at the Royal Irish Academy of Music.
In 2018, Science Foundation Ireland awarded academic geriatrician (and trained musician) Roman Romero-Ortuno the President of Ireland Future Research Leaders Award for the study of physiological signals to help the early detection of frailty in older adults [9]. This interdisciplinary research programme, called “FRAILMatics”, recruited researchers from physics, mathematics, and medicine, and involved the set-up of a new high-performance computer (HPC) cluster in Trinity College Dublin (TCD). As per TCD tradition, the new HPC system had to be named in honor of a distinguished scientist. On the suggestion of Louise Newman, biomedical engineer in The Irish Longitudinal Study on Ageing (TILDA), it was named “Tinney” in Sheila’s honor [10, 11].
To mark these developments and celebrate the work of both Sheila and her son Hugh, we created a short dynamic visualization titled “Dancing with Atoms”, combining a representation of Sheila’s ground-breaking work on crystal lattice structures with her son’s piano music, with the main processing of the visualisation performed on the Tinney HPC cluster. In the present work, we utilized the piece of music ‘Prelude in C major, BWV 846i (from The Well-Tempered Clavier, Book 1)’ performed by Hugh Tinney [12]. Three frequency bands were isolated: low, mid, and high. The magnitude change for these frequency bands were normalized to produce cartesian coordinates (x, y, and z), which were then used as inputs to an in-house developed atomic crystal lattice simulation, allowing the atoms to move (“dance”) in three-dimensional space in time with the music.
A. Audio processing
Audio processing was performed using MATLAB (R2020b; TheMathWorks, Inc, MA, USA). The audio source was an MP3 file with a sampling rate of 48kHz, bit rate of 192 kbps, and length 2 min 40 s. All processing was performed using in-built MATLAB functions. Firstly, both channels of audio (stereo) were combined into a single vector. A fast Fourier transform (‘fft.m’) was then used to convert this vector into frequency space and the real component of the data was isolated. Three new vectors were then produced, which isolated out three frequency bands: low frequency (‘sub-base’ to ‘lower midrange’; 16Hz-500Hz), ‘midrange’ (500Hz-2kHz), and high frequency (‘higher midrange’ to ‘brilliance’; 2kHz-20kHz). For each new vector all frequencies outside of these ranges were set to zero. An inverse fast Fourier transform (‘ifft.m’) was then used to convert the three vectors back into time signals. A quadratic regression smoothing function (‘smoothdata.m’) was then applied, with a window of 48000 (corresponding to 1 s for 48kHz data). All three vectors were then decimated to 60Hz and normalized to produce cartesian coordinates (x, y, and z), which were exported as a CSV file (along with a time vector) for use in the crystal lattice simulation.
B. Rendering a crystal lattice – Atoms and bonds
The rendering software used for this project was Trinity Visualisation Suite 2.0 (TVS2). This software was developed by Research IT, formerly known as the Trinity Centre for High Performance Computing (TCHPC; TCD, Dublin, Ireland) as part of the Institute for Information Technology and Advanced Computing (IITAC) project, with the goal of being able to visualize VASP files for the Trinity Computational Chemistry group. Over the years, additional functionality has been added to it, which makes it particularly well suited for adding custom features such as the ones examined in this project.
In this software, the rendering of a VASP dataset is based on the image of a single lattice cell. The dataset provides information about some of the atoms in the original cuboid, and it is up to the software to figure out which ones will be in the border of the same cell as it considers the neighbouring lattices. The bonds are then created by setting a distance interval that is larger than the Van Der Waals radii but is small enough that any pair of atoms within that distance must be bonded by either a covalent or Hydrogen bond.
C. Atoms moving independently
The atoms that were rendered can be, and often are, located at the boundary between two cells, so technically they belong to both (potentially, up to eight cells, if they are in one of the top or bottom corners of the lattice). Because of this, it was not possible to use a discrete number of copies of the lattice in each direction when choosing the magnitude of the movement. Instead, the distance to the original cell was used, so the distortion or perturbation became smaller as atoms were further away from the initial cell. Each one of these moved independently, which meant that all the atoms in the original cell moved with the original perturbation, with no scale or timestep distortion (i.e. all moving the same distance proposed by the current timestep), whereas the movement of atoms outside the original cell were affected by both perturbation and distortion. That way, the further away that an atom was from the original cell, the smaller the perturbation and the larger the distortion that was applied to it would be.
D. Bonds moving
Bonds were considered as links between two atoms within a predetermined distance interval. Thus, the movement of the bonds was determined by the movements of the two atoms that it linked. We acknowledge that this approach risks not preserving scientific accuracy - as both the perturbations and distortions applied to them could potentially allow a pair of atoms to move away from each other beyond the distance that would determine a chemical bond between them. However, for the sake of maintaining the crystal structure of the hypothetical material used for this visualization, bonds that were calculated from the unperturbed, undistorted crystal structure were retained throughout the animation, regardless of the distance between atoms.
E. Moving a single cell
All atoms in the original cell were perturbed by the amount determined by the CSV file extracted from the input audio data, which contained four columns (timesteps and x, y and z displacements), see Table 1.
| Algorithm 1: Draw original grid atoms and bonds |
|---|
| XYZ atomPosition = atomPositionthisAtom; XYZ perturbation = perturbationstimeStep; for Every atom in the original grid do glTranslatef(+ atomPosition + perturbation); drawAtom(); glTranslatef(-atomPosition-perturbation); end for for Every bond in the original grid do aPoint = atomPosition [ bondsbondId.atomA() ] + perturbation; bPoint = atomPosition [ bondsbondId.atomB() ] + perturbation; if atom Type of [bonds[ui].getX()] is different to atom Type of [bonds[ui].getY()]) then midPoint = (aPoint + bPoint)*0.5; setColorToAtom(bondsbondId.atomA()) drawBond(aPoint,midPoint); setColorToAtom(bondsbondId.atomB()) drawBond(midPoint,bPoint); else setColorToAtom(bondsbondId.atomA()) drawBond(aPoint,bPoint); end if end for |
F. Propagation along multiple cells - Space
Next, the propagation of the perturbation to atoms in other cells were calculated. In this case, the perturbation was reduced compared to the original cell as a result of distance from the original cell. Since, as previously mentioned, an atom can belong to more than one cell, two arrays were created, atomsDistance and atomsWeight. In atomsDistance the distance of each atom that was rendered to the original cell was stored. In atomsWeight, the weight to be applied to each atom was stored, depending on their corresponding value in atomsDistance (Table 2). These weights were then used to scale the perturbation of a particular atom, allowing for a smooth transition between cells (Table 3).
| Algorithm 2: Setting up atomsWeight |
|---|
| Resize atomsDistance to total number of atoms being rendered; for Every atom to be rendered do if This atom is in the origial grid then atomsDistancethisAtom = 1.0 else atomsDistancethisAtom = distance to the closes point of the original cell + 1.0 end if end for Resize atomsWeight to total number of atoms being rendered; for Every atom to be rendered do if This atom is in the origial grid then atomsWeightthisAtom = 1.0 else atomsWeightthisAtom= 1.0 / atomsDistancethisAtom end if end for |
| Algorithm 3: Draw atoms and bonds with spatial perturbation |
|---|
| XYZ atomPosition = atomPositionthisAtom; XYZ perturbation = perturbationstimeStep; for Every atom in the original grid do glTranslatef(+ atomPosition + perturbation * atomsWeightthisAtom); drawAtom(); glTranslatef(-atomPosition-perturbation * atomsWeightthisAtom); end for XYZ perturbationForAtomA = perturbationstimeStep * atomsWeightatomA; XYZ perturbationForAtomB = perturbationstimeStep * atomsWeightatomB; for Every bond in the original grid do aPoint = atomPosition [ bondsbondId.atomA() ] + perturbationForAtomA; bPoint = atomPosition [ bondsbondId.atomB() ] + perturbationForAtomB; if atom Type of [bonds[ui].getX()] is different to atom Type of [bonds[ui].getY()]) then midPoint = (aPoint + bPoint)*0.5; setColorToAtom(bondsbondId.atomA()) drawBond(aPoint,midPoint); setColorToAtom(bondsbondId.atomB()) drawBond(midPoint,bPoint); else setColorToAtom(bondsbondId.atomA()) drawBond(aPoint,bPoint); end if end for |
| Algorithm 4: Draw atoms and bonds with spatial perturbation and temporal distortion |
|---|
| XYZ atomPosition = atomPositionthisAtom; Int perturbationIndex = timestep - atomsDistancethisAtom; if perturbationIndex is less than 0 then perturbationIndex = 0; end if XYZ perturbation = perturbationstimeStep−perturbationIndex; for Every atom in the original grid do glTranslatef(+ atomPosition + perturbation * atomsWeightthisAtom); drawAtom(); glTranslatef(-atomPosition-perturbation * atomsWeightthisAtom); end for int perturbationIndexForAtomA = timestep - atomsDistanceatomA; XYZ perturbationForAtomA = perturbationsperturbationIndexForAtomA * atomsWeightatomA; int perturbationIndexForAtomB = timestep - atomsDistanceatomB; XYZ perturbationForAtomB = perturbationsperturbationIndexForAtomB * atomsWeightatomB; for Every bond in the original grid do aPoint = atomPosition [ bondsbondId.atomA() ] + perturbationForAtomA; bPoint = atomPosition [ bondsbondId.atomB() ] + perturbationForAtomB; if atom Type of [bonds[ui].getX()] is different to atom Type of [bonds[ui].getY()]) then midPoint = (aPoint + bPoint)*0.5; setColorToAtom(bondsbondId.atomA()) drawBond(aPoint,midPoint); setColorToAtom(bondsbondId.atomB()) drawBond(midPoint,bPoint); else setColorToAtom(bondsbondId.atomA()) drawBond(aPoint,bPoint); end if end for |
G. Propagation along multiple cells –Time lag
A time lag was also implemented as a function of distance from the original cell. The atomsDistance array was reused to determine how many timesteps to step back as the atoms got further away from the initial cell (Table 4).
H. Video conpositing
The atomic crystal lattice animation was then composited with the original audio source and the FRAILMatics logo added using Adobe Premier Pro (R2022; Adobe Inc., San Jose, CA, United States). The final video was then rendered in MPEG-4 format at 1562 x 932 (1.0) pixel resolution, 60 fps, progressive field order with 2-pass variable bitrate (VBR: target bitrate: 27 Mbps, maximum bitrate: 30 Mbps), audio: AAC 320 kbps, 48kHz, stereo.
Figure 2 shows plots of the raw stereo audio data, as well as the derived low (x-axis), medium (y-axis), and high frequency (z-axis) data. Figure 3 provides a screenshot taken from the final visualization, which is available to watch here: https://drive.google.com/drive/folders/15vXiQot4LXL5XA6H2-Op5T0aYS88KRxQ?usp=sharing (link also provided as a QR code at the end of this manuscript).
In this study, a short visualization titled “Dancing with Atoms” was produced in honor of Irish scientist Sheila Tinney, after whom a new HPC system was named at TCD. The visualization was inspired on Sheila Tinney’s work on crystal lattices, and consisted of an atomic lattice structure, which was perturbed in three-dimensions, using inputs derived from a musical piece by J.S. Bach performed by her son, Hugh Tinney. This creative signal processing project was mainly for the purpose of honoring Sheila Tinney’s work and to link it to the work of her son. Although the visualization may not be chemically accurate, it deeply resonates with FRAILMatics’ aim to promote collaborations in celebrating and promoting the work of women in STEAM.
There are several possible applications for this type of visualization, beyond the mostly artistic/visual purpose for the present work. One such application could be as a novel, visual hearing test, to test the frequency ranges that are audible for an individual. It is well known that, as we age, our ability to hear certain frequencies is reduced, and this is more pronounced for higher frequencies at older ages. For example, our hearing threshold for a 6kHz tone reduces in a fairly linear manor from the age of 50 to 100 years, however, our hearing threshold for a 0.5kHz tone is generally fairly stable, with a slight reduction from the ages of 50 to 80 years, but can rapidly reduce after the age of 80 years [13]. In order to use the visualization for this application, one would have to simply adjust the input x, y, z, and timestep data for the lattice visualization, potentially cycling through ever increasing frequency tones, with the lattice vibrating accordingly. The advantage to such an approach would be that it could be precise, with short steps of small frequency changes, and the test subjects would have a visual cue to know that there is a tone being played that they cannot hear.
Of note with the current work is that the approach taken was loosely based on a material sciences point of view, i.e., we imagined a hypothetical material, with a crystal lattice structure that would absorb and oscillate with wide and different frequency bandwidths along each cartesian axis (x, y, and z). This is obviously not a very plausible real-world scenario and was purely for visual purposes. However, such dynamic visualization of a live musical performance could be a potentially useful tool for training musicians; for example, the required smoothness of the sound (as in the current Hugh Tinney’s J.S. Bach performance) could be visualized through the movements of the lattice and perhaps provide an aid for training musicians to achieve the required sound by means of visual feedback. For example, one could ‘programme’ the lattice to ‘break’ above a pre-defined sound intensity (e.g., pianissimo) and/or when the correct tempo is not maintained.
There are several limitations to the current work. Firstly, an MP3 file was used as a basis for the audio processing, which is a lossy encoding format, meaning that some compression of the audio was implemented. The use of a lossless format, such as FLAC, would have provided an uncompressed audio source. However, the MP3 data were encoded at 192 kbps, which would have preserved most of the audible sonic information of the original recording. Another limitation is that higher end audio equipment (amplifier, speakers, etc.) would be required to reproduce the full frequency range (16Hz to 20kHz) correctly. However, it is a simple process to reduce the ranges used to accommodate the specifications of a particular system, as required.
In conclusion, the current study presented a method for extracting specified frequency ranges from audio data and using said data to simulate how an atomic crystal lattice structure might be perturbed in three-dimensional space by such soundwaves in a theoretical material. We dedicate this work to the life and work of Sheila Tinney, a truly inspiring and distinguished Irish scientist.
Competing interests
The authors declare no competing interests.
Funding
This study was funded by a grant from Science Foundation Ireland (SFI) [18/FRL/6188]. Calculations were performed on the Tinney cluster maintained by the Trinity Centre for High Performance Computing (Research IT); this cluster is funded form a Grant from Science Foundation Ireland under Grant number 18/FRL/6188.
- "Women in Science." The United Nations Educational, Scientific and Cultural Organization (UNESCO) Institute for Statistics. http://uis.unesco.org/en/topic/women-science (accessed 21st February, 2022).
- "Sheila Tinney." Wikipedia. https://en.wikipedia.org/wiki/Sheila_Tinney (accessed 21st February, 2022).
- S. C. Power, "Stability of crystals," Annexe Thesis Digitisation Project 2018 Block 18, 1941.
- H. W. Peng and S. C. Power, "On the stability of crystal lattices VIII. Stability of rhombohedral Bravais lattices," Mathematical Proceedings of the Cambridge Philosophical Society, vol. 38, no. 1, pp. 67–81, 1942, doi: 10.1017/S0305004100022234.
- W. Heitler and S. Power, "On the Origin of the Soft Component of Cosmic Radiation," Physical Review, vol. 72, no. 4, pp. 266–272, 08/15/ 1947, doi: 10.1103/PhysRev.72.266.
- S. Power, "The Intensity Distribution of Proper Vibrations," Proceedings of the Royal Irish Academy. Section A: Mathematical and Physical Sciences, vol. 49, pp. 91–100, 1943. [Online]. Available: http://www.jstor.org/stable/20488453.
- S. Power, "Decay of a Heavy τ-Meson into Three Lighter Mesons," Physical Review, vol. 76, no. 6, pp. 865–866, 09/15/ 1949, doi: 10.1103/PhysRev.76.865.2.
- S. C. Power, "Note on the Influence of Damping on the Compton Scattering," Proceedings of the Royal Irish Academy. Section A: Mathematical and Physical Sciences, vol. 50, pp. 139–142, 1944. [Online]. Available: http://www.jstor.org/stable/20520638.
- "Research into early detection of frailty in older adults awarded €1.5m." Trinity College Dublin, the University of Dublin. https://www.tcd.ie/news_events/articles/research-into-early-detection-of-frailty-in-older-adults-awarded-e1-5m/ (accessed 21st February, 2022).
- "Tinney HPC." Trinity College Dublin, the University of Dublin. https://tilda.tcd.ie/frailmatics/tinney-hpc/ (accessed 21st February, 2022).
- Trinity College Dublin, the University of Dublin. https://www.tchpc.tcd.ie/node/1353 (accessed 21st February, 2022).
- J. S. Bach, "Prelude in C major, BWV 846i (from The Well-Tempered Clavier, Book 1)," in Hugh Tinney: A Portrait, ed. Ireland: RTÉ lyric fm 2013.
- T. L. Wiley, R. Chappell, L. Carmichael, D. M. Nondahl, and K. J. Cruickshanks, "Changes in hearing thresholds over 10 years in older adults," (in eng), J Am Acad Audiol, vol. 19, no. 4, pp. 281 – 92; quiz 371, Apr 2008, doi: 10.3766/jaaa.19.4.2.