Introduction

Periodontal health is defined as the absence of inflammation in the periodontium (1). If gingivitis is not treated and properly controlled, it may progresses to periodontitis. In periodontal disease bacterial species interact with tissues and cells of the host, causing the release of a wide range of inflammatory cytokines, chemokines and mediators, which in some cases lead to the destruction of the periodontium (2).

Periodontitis is a multifactorial chronic inflammatory disease, and its most common cause is a dysbiotic biofilm (3). Dysbiosis is explained as disturbances in the composition of commensal communities relative to those present in health, and can occur as a result of a change in the microbiota or in the ability of the host to respond to changes. This delicate balance between homeostasis and dysbiosis is considered in part as an early training of local and systemic immune regulation (innate and acquired regulators) (4).

Bacteria in the oral cavity, tend to live in dental biofilms as highly complex and dynamic polymicrobial communities. Dental biofilm provides protection against shear forces and host immune responses. In a healthy individual, oral bacteria maintain a natural balance with the host, symbiosis, but different factors can contribute to this community becoming dysbiotic, allowing potentially pathogenic bacteria to increase in number and cause persistent infections such as periodontitis (5).

Several potential pathogens that promote the onset of periodontal disease have been identified (6). Different subgingival microbial complexes are known, which have been divided by colors according to the stage of development of the dental biofilm and the severity of the periodontal disease. Periodontal disease has been specifically linked to the colors orange and red. It is orange when species such as Fusobacterium nucleatum, Prevotella intermedia, Prevotella nigrescens, Peptostreptococcus micros (including associated species), Eubacterium nodatum, Campylobacter rectus, Campylobacter showae, Streptococcus constellatus, Campylobacter gracilis are present. As periodontal disease progresses, there is a shift to a red complex, which consists of the presence of Tannerella forsythia, Treponema denticola and Porphyromonas gingivalis. This bacterial consortium is strongly associated with the progression of periodontal disease, and the characteristics that unify the bacteria that comprise it are their extracellular proteolytic activities, complex anaerobic fermentation of amino acids, production of toxic metabolites, and their outer membrane vesicles (7).

Regarding red complex bacteria, it is known that virulence factors related to Porphyromonas gingivalis have an important role in coaggregation, biofilm formation and microbial dysbiosis, which can act directly on other bacteria or induce an optimal environment, including: capsule, outer membrane with its lipopolysaccharide, fimbriae, proteinases and selected enzymes (8). Few specific virulence factors related to Tannerella forsythia have been identified, including: trypsin-like proteases and PrtH, SiaHl and NanH sialidases, Leucine- Rich Repeat BspA Protein, α-D-glucosidase and N-acetyl-β-glucosaminidase (9), a surface S-layer, and activity for the induction of apoptosis. Of the known virulence factors of T. denticola the mainly mentioned are the production of hydrolytic enzymes such as hyaluronidases, collagenases, proteases, phospholipases and phosphatases; others such as, Msp, OppA, FH-like binding protein, dentilisin, and leucine-rich repeat proteins (LrrA), are also considered (10).

Periodontal disease affects a large part of the population, it can occur in children, adolescents, adults and older people. Many times clinical, surgical and drug treatment is not enough to stop the progression of the disease. Therefore, it becomes essential to know the intrinsic and extrinsic factors that can lead to its appearance, degree and extension, as well as to know which of these factors affect the most at certain stages of disease development. Susceptibility is a host condition, influenced by a series of environmental and intrinsic factors that increase their chances of developing periodontal disease.

Genetics has been shown to be a risk factor for the development of periodontal diseases, with variability of up to 1/3 in a population. Patients who develop periodontal disease early in life commonly have genetic risk alleles (11). Alterations in the immune system have also been shown to influence the prevalence of periodontal diseases in children and adolescents (12). Stress has been proposed as another risk factor, since the high levels of hormones present in stressful situations could be associated with a poor host immune response and an increased inflammatory response (13).

External factors such as diet, oral hygiene and access to oral health services are fundamental determinants in the development of periodontal diseases (14).

Clinical studies have been conducted in different parts of the world demonstrating alarming figures regarding the prevalence of gingivitis in children and adolescents. For example, Owino et al. (2010) analyzed a group of 292 students from Kitale, Kenya. Based on the Community Periodontal Index, the researchers determined that the prevalence of gingivitis was 77.7% (15). In a study in Asia by Ballouk and Dashash (2018), 1500 boys and girls were analyzed using the Silness and Löe Gingival Index, and found a prevalence of gingivitis of 97.93% (16). In Puerto Rico, a research carried out by Elías et al. (2018) with a sample of 1586 12-year-old schoolchildren, showed a prevalence of gingivitis of 80.41% (17). An epidemiological analysis in Costa Rica in 2017, on the oral health of 624 adolescents, revealed that 25% of the participants had periodontal disease (18). Also in Costa Rica, in a study carried out on 12-year-old children in 2019, indicated that 96.8% of the participants had gingivitis (19).

The presence of periodontopathic microroganisms, such as Prevotella nigrescens, Trepo- nema denticola, Aggregibacter actinomycetemcomitans and Porphyromonas gingivalis, were detected in periodontally healthy children aged 6-13 years old (20). It seems the colonization of putative periodontopathic bacteria can occur early in childhood without clinical signs of periodontitis.

Few studies have been carried out world-wide, assessing the prevalence of periodonto- pathogenic bacteria in 12 year-olds. It is important to study this population group, 12 year-olds, since it is the average age that marks the beginning of adolescence and it is when most of the permanent dentition appears. Knowing the gingival status of this population will help dentistry professionals to focus more on education, health promotion and disease prevention, in order to reduce clinical intervention in the future. In addition, the presence of periodontopathic bacteria in children's saliva is of interest, since it may be a risk indicator for the transmission, development and control of periodontal disease. Therefore, the objective of this investigation, was to evaluate the gingival state and the presence of red complex bacteria of 62 12-year-old schoolchildren from Carmen Lyra School in Alajuelita, San Jose, through a clinical examination and polymerase chain reaction (PCR) analysis of salivary samples.

Methods

This cross-sectional study was approved by the Institutional Review Board of the University of Costa Rica (VI-5628-CEC-0007-2018).

Data collection was carried out between June and July 2019 in 62 (31 boys and 31 girls) 12-year-old students of the public Carmen Lyra school, located in Concepción Arriba de Alajuelita, San José, Costa Rica. Parents or guardians signed a written informed consent and children provided written assent.

Clinical examination

Clinical examination to determine the gingival state of the participants was carried out in the dental clinic of the Carmen Lyra School by a calibrated periodontist (KRC). Each participant was sat in the dental chair and the examination was performed using the angle lamp of the dental chair as light source.

To determine the biofilm index (BI), (Silness and Löe Index of 1964 (21)); the buccal and palatal/lingual surfaces of six teeth were evaluated. The teeth selected for evaluation using the FDI two-digit notation were 16,11,26,46,41, and 36. Due to the age of the participants, the first molars of the four quadrants were used instead of the premolars, since the first molars had already been in the participants' mouth for at least 6 years.

Gingival health was determined using Silness and Löe Index of 1967 (22) using a WHO (World Health Organization) periodontal probe, which has a 0.5mm diameter ball in its active part. The teeth examined were also 16,11,26,46,41 and 36 on the six dental surfaces (mesial, medial, distal, buccal, lingual/palatal) to assess the presence of gingival bleeding.

Gingivitis was defined as the presence of bleeding on probing in at least one site, and the extension was classified according to the percentage of teeth whose gingiva presented bleeding on probing. Thus, from 25 to 49% of the teeth tested, was classified as limited gingivitis, and extensive gingivitis if more than 50% of the teeth tested had bleeding o probing.

Subsequently, each participant was evaluated in order to determine the presence of dental calculus (supragingival, subgingival or both) in all the teeth present in their mouth.

Before saliva collection, each child was given a toothbrush, toothpaste, and given oral hygiene instructions according to Stillman brushing and spinning technique. The participants practiced and brushed their teeth with water only.

Saliva collection

The 62 saliva samples were taken in 50ml polypropylene tubes. Each participant was asked to rinse their mouth with a prepared solution of Listerine® and water (20ml of Listerine+20ml of sterile double distilled water for each sample). After the first mouth rinse, each boy and girl mouth rinsed one more time, both rinses with 20ml of Listerine® 1:2. Each rinse lasted approximately one minute. Listerine® was diluted in sterile double distilled water, to avoid burning the participants' mouths; and, in addition, the dilution decreases the alcohol concentration by approximately 10%. This is important since a high percentage of alcohol in the samples could reduce bacterial growth, when they were stored.

After rinsing, the samples were processed for DNA extraction, following a modification by one of the authors of this publication.

DNA isolation process of saliva samples

Samples were centrifuged at 2920 rpm at 4°C for 30 minutes (in the Eppendorf® Centri- fuge 5810R. The supernatant of each sample was decanted and resuspended in 10ml-15ml of sterile PBS and each mixed gently with Vortex Mixer VM-300 serial number 908777 (Gemmy Industrial Corp.). The samples were centrifuged at 2920 rpm (Eppendorf® Centrifuge 5810R) at 4°C for 30 minutes. Next, each sample was decanted in a beaker with sodium hypochlorite (3.5%) and each one was resuspended with lysis buffer + high TE:

500μl of lysis buffer + high TE were added to samples: 6,8,10,12,13,14,15,16,18,19,20,24, 25,27,30,31,32,34,36,37,39,40,41,42,45,48, 50,53,54,55 and 62.

1000μl of lysis buffer + high TE were added to the samples: 1,2,3,4,5,7,9,11,17, 21, 22, 23, 26, 28, 29, 33, 35, 38, 43, 44, 46, 47, 49, 51, 52, 56, 57, 58, 59, 60 and 61.

Buffer concentrations depend on the size of the button of cells that forms at the beginning of the extraction; for example, if the button is large, 1000μl of lysis buffer+high TE is used, but if it is small, 500μl of lysis buffer+high TE is used.

25µl of proteinase K (20mg/ml) was added to each sample and the samples were incubated overnight in a 57°C water bath (Precision Water Bath, GCA Corporation (37383)).

The samples were removed from the water bath and the same volume of 3M NaCl, which was used as total lysis solution, was added to each one:

500 μl of 3M NaCl was added to samples: 6,8,1 0,12,13,14,15,16,18,19,20,24,25,27,30,31, 32,34,36,37,39,40,41,42,45,48,50,53,54,55 and 62.

1000ul of 3M NaCl was added to samples: 1,2, 3,4,5,7,9,11,17,21,22,23,26,28,29,33,35,38, 43,44,46,47,49,51,52,56,57,58,59,60 and 61.

Subsequently, an equal volume of chloroform-isoamyl alcohol (24:1) was added to each sample. Samples were mixed in 211DS Shaking Incubator, Labnet, at 235rpm for 1 hour. Each of the samples was transferred to a 15ml polypropylene tube and centrifuged at 1800g, 4°C for 5 minutes.

Next, the aqueous phase of each sample was transferred to a new 15ml tube and 2.5 volumes of absolute ethanol were added to the respective aqueous phase of each sample (µl of each sample x 2.5 volumes of ethanol). Each tube was labeled and carefully checked to avoid sample mix-ups.

The tube was gently inverted until the DNA strands appeared. Initially, a pellet did not appear in any of the samples, so the samples were kept at -20°C for 14 days. After that time, the samples were centrifuged at 1800 g for 30 minutes, decan- ted and 1ml of 70° ethanol was added to each.

After this step, the samples were kept at room temperature overnight.

The following day, the samples were centri- fuged at 1800 g at room temperature for 10 minutes. They were decanted and placed in a rack with the lid open and allowed to dry for 2 days at room temperature with a paper towel on top.

Samples were resuspended in 375µl of TE low EDTA buffer (Invitrogen, Catalog No.: 12090- 015). The samples were then incubated overnight in a water bath at 37°C.

The NanoDrop 2000C spectrophotometer (Thermo Scientific (332045) was used to evaluate the quality and quantification of the samples. Finally, they were stored at 4°C.

PCR: 16S gene assay

Samples were diluted to a concentration of 25 ng/µl and assayed with primers of the 16S gene as positive control, since all bacteria have the 16S gene.

Each sample was prepared using the following reagents: 7.5µl of MM2X, 3.5µl of water, 2µl of primers, and 2µl of DNA template or water (negative control).

A modified amplification program was carried out for the analysis of the samples.

Samples 2,4,7,8,18,19,20,21,22,28,31,32,36,39,41,43,48,53,60,61 were negative with the 16S gene primers in the first assay; therefore, the assay was repeated with the same samples undiluted (that is, at the original concentration of the DNA extraction).

In the second assay, samples 2,4, 8,31,32, 39 and 41 were again negative with the 16S gene primers; therefore, the PCR assay was repeated again with these samples and in the same way with the original concentration.

In the third assay, sample 4 was the only one to test negative with the 16S gene primers.

Subsequently, we decided to withdraw samples 4,8,18,39 and 41 from the project, since they were negative for 16S gene primers.

For the PCR assays with Porphyromonas gingivalis, Tannerella forsythia and Treponema denticola the procedure performed with the 16S gene was repeated.

Electrophoresis gel

The gel was made with a concentration of 1.5% agarose in 0.5X TBE. A 5μl volume of amplification product was used with 1μl of loading dye with GelRed, 4μl of the GeneRuler 1kb Plus DNA Ladder (Thermo Scientific), and the gel was subjected to a constant 100-volt electric current for 45 minutes. Finally, by means of an UV transilluminator, the bands for the DNA amplification results for Tanerella forsythia were visualized.

Analysis of the results

The resto of amplification results were obtained using a QIAxel high-resolution capillary electrophoresis device.

Statistical analysis was conducted using IBM SPSS STATISTICS version 22, to analyze variables on gingival health. Normality and homogeneity of the data was confirmed. Frequencies were obtained and crossing of variables were made, variance of means were calculated.

Results

Biofilm index

The mean BI according to Silness and Löe (1964) (21) was 1.18, this means that the presence of biofilm is not detectable with the naked eye, but it can be seen by passing the periodontal probe over the dental surface. No statistically significant differences in BI were found between girls and boys (p=0.322); however, it was possible to observe that boys obtained a higher BI (1.24) compared to girls (1.12).

Dental calculus

The prevalence of dental calculus was 40.40%. Supragingival calculus was found in 19.4% of participants and 21% had both supragingival and subgingival calculus. No statistically significant difference (p=0.160) was found between the average number of teeth with dental calculus between boys (1.30) and girls (1.41).

Regarding the presence of dental calculus, when considering the number of teeth with calculus and the number of bleeding surfaces (p=0.0009), a significant relationship was established with the presence of bleeding on probing.

Gingival state

When evaluating the gingival state, it was found that 96.8% of the participants presented gingivitis; 11.5% of schoolchildren had limited gingivitis and 88.5% extensive gingivitis. According to the Silness and Löe Gingival Index, the children obtained an average index of 0.97, which means that there was a predominance of mild gingivitis. According to this criteria, a result of 0.1-1.0 is classified as mild gingivitis, 1.1-2.0 as moderate gingivitis, and 2.1-3.0 as severe gingivitis.

No statistically significant difference (p=0.639) was found between the average number of teeth with gingival bleeding in boys (1) and in girls (0.94).

Presence of red complex bacteria

Through gel electrophoresis and the QIAxcel device, the results presented in Table 2 were collected to determine the presence of the bacteria Treponema denticola, Porphyromonas gingivalis and Tanerella forsythia in 57 of the samples collected, since samples 4,8,18,39 and 41 were withdrawn from the project, as they were negative for the 16S gene primers.

According to the results obtained, 32 samples (56.14%) showed a positive result for the presence of Treponema denticola, 18 samples (31.58%) were positive for Tanerella forsythia and 7 samples (12.28%) were positive for Porphyromonas gingivalis.

Treponema denticola was the most frequent in the samples (Table 2), while the least frequent in the samples was Porphyromonas gingivalis. Among the total number of participants, 18 of the samples (31.58%) did not present any of the bacteria analyzed and the remaining 39 samples (68.42%) presented a positive result for the presence of at least one of the bacteria of the red complex.

Of the 10 samples in which two of the three bacteria were present, 8 of them (80%) were positive for Treponema denticola and Tanerella forsythia, 2 of the samples (20%) were positive for Porphyro monas gingivalis and Treponema denticola. In none of the samples analyzed was a combination found with the presence of Porphyromonas gingivalis and Tanerella forsythia.

And finally, of the 25 samples that were positive for only one of the bacteria, 18 of them (72%) were positive for Treponema denticola, 6 (24%) were positive for Tanerella forsythia and 1 of the samples (4%) tested positive for Porphyromonas gingivalis.

Discussion

According to the results obtained related to BI and the presence of calculus, it is probable that the subjects had an inadequate oral hygiene at the moment of the clinical examination and that the main cause for gingivitis was the accumulation of biofilm. In our study, the reported BI was 1.18, with 96.8% of the participants presenting gingi- vitis (11.5% of schoolchildren presented limited gingivitis and 88.5% extensive gingivitis) and mild gingivitis predominated. A similar relationship was reported by Ballouk and Dashash in 2018, who reported finding a biofilm index of 1.12 and a prevalence of gingivitis of 97.93% (16).

In this study, 40.40% had calculus (21% both subgingival and supragingival), without statistically significant differences between boys and girls. This percentage is lower compared to other studies conducted in 12-year-old children from other countries such as Puerto Rico (61.59%) (17), China (66.94%) (23), and Ecuador (60.5 to 77.9%) (24). Variations may be attributed to differences in methodology of the studies and may also suggest or reflect differences in oral hygiene practices. What is known is that calculus is a key factor in the development of periodontal disease, since dental calculus triggers inflammatory reactions in supporting tissues of teeth (25).

Among the total number of participants and according to the PCR tests for red complex bacteria, 18 of the samples (31.58%) did not present any of the bacteria analyzed, and the remaining 39 samples (68.42%) presented a positive result for the presence of at least one of the bacteria of the red complex. Our results confirm the presence of red complex bacteria among the population studied, whose gingival health was also affected, being an indicator of the effect of dysbiosis caused by the red complex bacteria analyzed and found in the samples.

Red complex bacteria are characterized by promoting the progression of periodontal disease (26, 27) and are late colonizers in the dental biofilm; therefore, its presence is an indicator of an already established dysbiosis that affects the gingival health of the host. Based on the gingival health status of the subjects, it is thought that there is a relationship by finding the presence of Treponema denticola, Tanerella forsythia, and Porphyromonas gingivalis and a diagnosis of gingivitis with indicators of periodontal disease such as inflammation, bleeding gingival and presence of calculus.

Regarding the relationship between the BI, dental calculus and the incidence of gingivitis with the presence of red complex bacteria in saliva samples, it is known that if the individual has a symbiotic oral environment, the bacteria remain in balance; but if this environment is altered by factors such as biofilm and calculus accumulation, the oral environment becomes dysbiotic and pathogenic bacteria proliferate, which can lead to periodontal diseases (26,27,28).

As reported previously (29), we found an interaction between two of the red complex bacteria in the samples, 80% was positive for Treponema denticola and Tanerella forsythia, and 20% for Treponema denticola and Porphyromonas gingivalis. It has been demonstrated Treponema denticola binds to Porphyromonas gingivalis and Tanerella forsythia with similar avidity (29). This may explain the close association Tanerella forsythia exhibited with the other two red complex bacteria, thus reiterating this theory. It has also been demonstrated that LrrAs proteins of Treponema denticola intervene with binding to Tanerella forsythia, but not Porphyromonas gingivalis. Additionally it is important to mention, LrrA proteins are important in epithelial cell invasion and biofilm formation by Porphyromonas gingivalis (29). Due to this virulence factor that Treponema denticola posses- ses, the predominant association of this bacterium with Tanerella forsythia and Porphyromonas gingivalis, in our study, could be explained.

Treponema denticola is usually found in sites with the greatest depth of the periodontal pocket and, in healthy ones, its presence is rarely verified, since it is not an early colonizer of subgingival biofilm (7). Treponema denticola was the most prevalent bacteria in the samples we analyzed. Basic research as well as clinical evidence suggest that the prevalence of Treponema denticola, together with other gram-negative bacteria in high numbers in periodontal pockets, may play an important role in the progression of periodontal disease (30).

Tanerella forsythia has been frequently associated with various forms of periodontal disease, since it was found that this microorganism forms synergistic biofilm mixtures with F. nucleatum (considered a ''bridge'' bacterium due to its ability to coaggregate with early and late colonizing bacteria), which facilitates biofilm formation (7). Porphyromonas gingivalis also plays an important role in coaggregation, biofilm formation and microbial dysbiosis, because they can act directly on other bacteria or induce an optimal environment for the development of periodontal disease (31).

Porphyromonas gingivalis and Treponema denticola were the second most prevalent interaction in the samples, these bacteria are frequently found in periodontal lesions. Treponema denticola an oral spirochete, is found in the superficial layers of the subgingival biofilm. Porphyromonas gingivalis is predominantly observed under a layer of spirochetes, which demonstrates the symbiotic relationship that exists between both bacteria (32).

A third interaction is the one that occurs between Tanerella forsythia and Porphyromonas gingivalis, especially in an active state of periodontitis. It has been suggested that a nutritional symbiosis occurs between them (32). In this study, the samples did not present an interaction of this type, which allows us to infer that the periodontal disease of the participating children is not in an active state, or rather, it is in early stages.

The indexes and clinical parameters used in this study are related to the prevalence of gingivitis. In the other hand, the laboratory analyzes our group performed on the saliva samples, determined the presence of red complex bacteria. As mentioned before, these microorganisms are directly related to periodontitis.

The determination of the depth of the periodontal pocket is an important element to determine and classify periodontitis, however, this factor was not considered in this work, a limitation of this study. Intrinsic factors related to periodontal disease such as genetic alterations, stress, autoimmune diseases, diastolic pressure and diabetes in children and adolescents (33) are factors that were not considered in this study either. Neither the level of education of the parents nor their hygiene habits were taken into account, whose influence on the oral health of the participants was decisive.

One of the strengths of this study is that the small size of the recruited population allowed clinical parameters and the collection of saliva samples to be carried out by a single calibrated operator, a specialist in Periodontics. Thus, reducing the chances of bias and providing greater specificity. Another of the strengths of the study was the participants had many things in common. For instance, they were 12 year-old boys and girls from the same geographical area, the same school level, similar access to similar oral health services (provided in the same school) and belonging to the same educational center. All these variables provide sample homogeneity and provided information regarding the environment in which the participating subjects lives. These information reduces the possibility of socioeconomic and sociodemographic differences among them.

The socioeconomic situation and the possibility of having regular access to health services are important factors to consider regarding the gingival and dental health status of children. The Carmen Lyra School in association with the Faculty of Dentistry of the University of Costa Rica provides free dental services to its students; however, the restorative intervention turns out to be not enough to control periodontal disease and the presence of red complex bacteria that are associated with it. For this reason, education in health promotion and disease prevention, such as individualized indications for correct oral hygiene, both for students and parents/guardians, are important to prevent periodontal disease, in addition to access to dental health services.

This study provides information of interest and importance to carry out future studies that analyze other factors that could influence periodontal disease in young population, both extrinsic and intrinsic, and relate them to the results obtained; so that it is determined if the presence of bacteria of the red complex together with an adequate diagnosis of periodontitis, present a higher prevalence in subjects who are exposed to these variables.

Conclusion

We identified a substantial percentage of red complex bacteria in the saliva samples, and a diagnosis of generalized gingivitis in the students. It can be deduced that there is very likely relationship between the presence of these bacteria and the affected gingival health of the schoolchildren that participated in the study. It has been suggested that high plaque retention in the oral cavities of children seems to promote the colonization of periodontal pathogens (34).

The biofilm index obtained, and the presence of calculus, demonstrate a poor hygiene technique in the participants, which is one of the main factors for the development of periodontal disease. The presence of bleeding is also indicative of inflammation and accumulation of dental biofilm, which reaffirms the diagnosis obtained in the participating subjects. Bleeding on probing remains the best parameter to monitor gingival health or inflammation longitudinally, and the practice of probing should be initiated once permanent first molars are fully erupted. The importance of an assessment of the periodontal status of pediatric patients should be part of a routine dental visit and oral examination.

Despite being a population with free and regular access to dental services, it has been shown that this is not enough to guarantee good gingival health. It is advisable to emphasize education in oral health promotion and disease prevention directed specifically at parents and guardians of students at the Carmen Lyra School and thus achieve adequate control of biofilm formation.

To achieve greater preponderance in a similar study, it would be beneficial to consider the presence and depth of periodontal pockets in children with permanent dentition, as parameters to be studied; since it would help to determine the severity of the periodontal disease present in the participating subjects and its result would generate a greater impact.

Funding

Universidad de Costa Rica Research Funds given to KR (project B9301) and SSdlF (project B9468).

Conflict of interest

None

Author contribution statement

Conceptualization and design: K.R. and S.S.dlF.

Literature review: L.O.A., A.M.G. and R.R.M.

Methodology and validation: K.R., S.S.dlF. and A.V.Ch.

Formal analysis: K.R. and A.V.Ch.

Investigation and data collection: K.R. and A.V.Ch.

Resources: K.R., A.G.F., S.S.dlF. and A.V.Ch.

Data analysis and interpretation: K.R. and A.V.Ch.

Writing-original draft preparation: L.O.A., A.M.G., R.R.M.

Writing-review and editing: K.R., A.G.F., S.S.dlF., A.V.Ch.

Supervision: K.R., A.G.F., S.S.dlF., A.V.Ch.

Project administration: K.R.

Funding acquisition: K.R., S.S.dlF.

Acknowledgements

We would like to thank the administration at Carmen Lyra School for helping us conduct this study.

References

Publication Dates

History