Abstract:
The objectives of this studies were to create Paper-Based and Internet-Based Digital Media Literacy Situation (DMLS) Tests and examine their psychometric properties, to analyze the effects of the created Paper-Based and Internet-Based DMLS-tests, and to compare the effectiveness of Paper-Based and Internet-Based DMLS-tests. The samples were 393 high school students under the Secondary Education Service Area Office 1 and 2 in Bangkok during the academic year 2018. The research instruments included the created Paper-Based and Internet-Based DMLS-tests and a questionnaire on the satisfaction of students towards the use of DMLS-tests. The collected data were analyzed using descriptive and inferential statistics, including frequency, percentage, means, standard deviation, skewness and kurtosis. The psychometric properties were analyzed with regard to reliability, validity, difficulty, and discrimination, while the confirmatory factor analysis was conducted with respect to mean and standard deviation. The results were as follows: 1. Considering the psychometric properties, the Paper-Based and Internet-Based DMLS-tests are similar in terms of content validity, construct validity, internal consistency, difficulty, and discrimination. The Paper-Based had good content validity (IOC=0.80 1.00), while the pBT-CFA model was fit to the empirical data (chi-square=24.75, p=.363, df=23, CFI=1.00, SRMR =.03, RMSEA=.01). As its Cronbachs Alpha was 0.743, its average difficulty (b) was -1.96 and its average discrimination (a) was 1.28. The Internet-Based had good content validity (IOC=0.80 1.00), while the iBT-CFA was fit to the empirical data (chi-square=35.59, p=.061, df=24, CFI=.99, SRMR=.03, RMSEA=.04). As its Cronbachs Alpha was 0.741, its average difficulty (b) was -1.95, and its average discrimination (a) was 1.23. 2. According to the examination of measurement methods of the DMLS-tests using the CTCM model, it was found that the CTCM model was fit to the empirical data (chi-square2 = 286.40, p=.000, df=128, CFI=.97, SRMR=.05, RMSEA=.06). Comparing the factor loadings resulted from digital media literacy and measurement methods, there were 2 indicators (10%) ensuring that the factor loading of the measurement methods was greater than the factor loading of the digital media literacy. This meant that the effect size of measurement methods of the digital media literacy situation tests was small. 3. In terms of student satisfaction, the difference between the Paper-Based and Internet-Based DMLS-tests was not statistically significant at the .01 level. The average level of student satisfaction towards the Paper-Based test was 4.42 (SD=0.57), while the average level of student satisfaction towards the Internet-Based test was 4.42 (SD=0.56). In terms of the average test-taking time, students took less time to finish the Internet-Based test. In addition, the Internet-Based test had advantage over the Paper-Based test in terms of grading and interpretation of test scores as the test results were available immediately after test completion.