Python Data Set Generation and R Data Analysis


As a way to show my capabilities regarding computer programming that would be relevant to my desire to work on psychology research, which most notably means the R software. However, I would want to do so with a data set that I could use in order to show what I can do, which runs into an issue of me not knowing where to find such a thing, I elected to make one of my own, and I decided to do so by making a different program for doing so in Python. So, I want to go over my thought process in making the initial programs I have for both.

Python: Randomizing Data Sets

from random import randint \n import csv

To start with, the Python code I made to create test data sets, the key packages I needed for both were the random and the csv. I only needed the randint function from the former, so I imported only that into the file, while my choice for using the csv package is more so a result of not being able to download a package in order to export the results as a Excel file, which could be imported into R to more easily work with. However, this mostly ends up just being a minor inconvenience, as a csv file can be converted into an Excel file [or more accurately a .xlsx file] via a spreadsheet program like Microsoft Excel, Google Sheets, or LibreOffice Calc.

def partinfo(partnum, sigcond, totalparts): \n partname='Participant ' + str(partnum) \n if partnum<=totalparts/2: \n cond=1 \n elif partnum>totalparts/2: \n cond=2

The primary function that allows for everything to work is the admittedly poorly named partinfo function, which takes as variables partnum, sigcond, and totalparts as variables. The first line written is just to create what amounts to an anonymized title for a participant. The next four are just for assigning the non-existent participant to one of the two conditions, which also results in the program not working with odd numbers. I also created two other versions of this program; one that has three conditions, though only works with numbers divisible by three; and one that can take any number of conditions, but it ends up with entirely random numbers of individuals assigned to each condition.

lowreroll=0 \n highreroll=0 \n while highreroll<2 and lowreroll<3: \n grade=randint(40,100) \n if grade<60 and cond!=sigcond: \n lowreroll+=1 \n elif grade<70 and cond==sigcond: \n lowreroll+=1 \n elif grade>95 and cond!=sigcond: \n highreroll+=1 \n else: \n break \n scond=str(cond) \n sgrade=str(grade) \n wstudy=str(randint(0,6)) \n return [partname, scond, sgrade, wstudy]

The next series of lines is meant to do two things. First, it is meant to keep most of the data above failing for the most part, since it is supposed to be a variable used in this faux data set, while still allowing it to be below 60 on rare occasions. Second, it is meant to allow for some of the data to be significant if that is intended to be the case by making it harder for the non-significant condition to have a grade above 95. I also made it only re-roll above a 95 once for a non-significant condition, resulting it only happening 0.7% of the time compared to about 8.3% in a significant condition, while rolling below 60 should only happen about 3.6% of the time for both. In hindsight, I probably should have made this even lower somehow, but I digress. Then convert the variables into a string version for later use and generate weekly study time as another variable, and then returning them for use in the main part of the function.

def main(): \n totalparts=int(input('How many participants? (Even numbers only): '')) \n ifsig=input('Do you want a condition to be significant (y/n): ') \n sigcond=3 \n if ifsig=='y': \n sigcond=randint(1,2) \n print(str(sigcond)) \n partnum=0 \n fields=['Name', 'Condition', 'Grade', 'Weekly Study Time'] \n rows=[] \n while partnum<totalparts: \n partnum+=1 \n pinfo=partinfo(partnum, sigcond, totalparts) \n rows.append(pinfo) \n print(rows) \n fname=input('Filename: ') \n fname=fname + '.csv' \n with open(fname, 'w') as csvfile: \n csvwriter = csv.writer(csvfile) \n csvwriter.writerow(fields) \n csvwriter.writerows(rows)

The main function, while maybe not needed due to only having one function, is something I am used to creating as separate from some part of the overall program due to habit and in case I think I would need to use make other functions. Plus, I find it to be the best place to put all of of the inputs into. Speaking of which, the initial one just asks for the number of participants for the data set and the if the data set should have a significant condition, with it being set to 3 here initially to prevent it from being used later on. The other versions I brought up earlier have a similar set up, but have the sigcond variable set to different values for various reasons. Also, in the purely random one, you can put in any number of conditions. Then an if statement for randomizing the sigcond if asked for, as well as printing it if doing so you know how the data is supposed to look when processed. Then some lists for the creation of the csv file and a while loop to use the partinfo function for, while increasing the value of partnum each time to create the participant number and to assign them to a condition in the partinfo function, with me doing the while loop the way I did to make sure I would not cause an issue with the first or final values. Then assign the output to the earlier rows list, print said list to make sure the data has actually been assigned, and then export everything into a csv file with everything else, with the filename being the result of another input function with an added .csv to make sure it gets exported properly. Then I convert the csv file into a .xlsx and then we move onto R.

R Codes: Analyzing Data Sets By Conditions

Multi_Mean_SD_Calc<-function(dsn, dscon, dsd){ \n #dsn=dataset name, dscon=dataset condition, dsd= dataset data

Having now converted the .csv file into a .xlsx one, we can now import the data set into R. I named my file datasettest for sake of clarity. Then I defined the name of the function I will be using for the purposes of finding the mean and standard deviation for both the whole of the data set and for each of the conditions when it comes to the grades variable.

tmean<-mean(dsd) \n tsd<-sd(dsd) \n rtsd<-round(tsd,digits=3) \n #Calculate Total Mean and SD

Then for the start, I calculated the mean and standard deviation for each in order to get both for the total data set, with me rounding the standard deviation to the first three digits for aesthetic purposes later, but it can be removed to show the whole of it.

conMean<-aggregate(x=dsd,by = list(dscon),FUN=mean) \n sd1<-aggregate(x=dsd,by = list(dscon),FUN=sd) \n rsdc1<-round(sd1$x[1],digits=3) \n rsdc2<-round(sd1$x[2],digits=3) \n #Calculate Mean and SD for Each Condition

The next step is to find each of the means and standard deviations for the two conditions. The aggregate function I use here does do what I want to make the raw numbers, but ends up saving said number in a data frame. As such, in order to call said number, I have to specify what variable I need from the data frame. Here, I elect to do so know with the standard deviations and using them to define a new variable, mostly to make it easier to round them and use the rounded version later.

mmsddf<-data.frame(Mean=c(tmean,conMean$x[1],conMean$x[2]), \n St.Dev.=c(rtsd,rsdc1,rsdc2), \n row.names=c('Total','Cond. 1','Cond. 2') \n ) \n #Put it all into a Data Frame for East Comparisons \n print(mmsddf) \n } \n\n Multi_Mean_SD_Calc(datasettest, datasettest$Condition, datasettest$Grade)

Final step is to put everything into a newly defined data frame, listing the rows by name for the total and the two conditions, alongside listing which one is the mean or the standard deviation, which is why I rounding earlier. The print function is just to show the output, though if you were to run the code line by line, you would get the data frame mmsdf (Multi_Mean_SD_Dataframe) as something you can view outside of the print function. Then the final line is just calling the function and defining what variables are used to then use the function.

Mean: Total: 77.07 | Cond. 1: 78.34 | Cond. 2: 75.80 St.Dev: Total: 14.429 | Cond. 1 15.772 | Cond. 2 12.984

However, I also did work on a number of other versions of this same program, most notably one that I use with the version with the fully randomized version of the Python data set generator, which feature four conditions. I mostly just want to highlight some of the differences in between these two versions.

mlist<-vector(mode='list') \n sdlist<-vector(mode='list') \n rnlist<-vector(mode='list') \n #Defining lists for later \n \n mlist[length(mlist)+1]<-tmean \n sdlist[length(sdlist)+1]<-rtsd \n rnlist[length(rnlist)+1]<-'Total' \n #Hey, its later, adding means and SDs to lists, as well as names

The first big difference are the lists. These are for later with the data frame in order to make things easier as this version is meant to be able to work with versions of the dataset regardless of how many conditions I create. Granted, said conditions are solely whole numbers, but I could easily make adjustments in order to focus around situations involving values above or below specified values.

conMean<-round(aggregate(x=dsd,by = list(dscon),FUN=mean), digits=2) \n sd1<-aggregate(x=dsd,by = list(dscon),FUN=sd) \n #Outcomes for each group of conditions \n \n conC<-1 \n maxCon<-max(dscon) \n #Variables relevant for counting \n \n if (conC!=maxCon){ \n #Needed to prevent situations where only one condition specified \n while (conC<=maxCon){ \n #Conditions for loop \n mlist[length(mlist)+1]<-conMean$x[conC] \n rsdc<-round(sd1$x[conC],digits=3) \n sdlist[length(sdlist)+1]<-rsdc \n #Adding variables to list \n narow<-paste('Cond. ', conC, sep=' ') \n rnlist[length(rnlist)+1]<-narow \n \n #Creating additional labels for the rows for the Data Frame later, and adding them to the list \n conC<-conC+1 \n } \n }

The next big difference is the if statement and the while loop. The dscon value is only really needed for the if statement to box out times when there is only one condition, though that can cause issues, so it definitely could be improved, though to be fair the number of situations when having a program be able to work with an undefined number of conditions, at least within the context of a R program for a research project, are few and far between. Regardless, the while loop makes great use of the conC variable, both to limit the loop in order to prevent from going on to infinity, and to find and add the values from the aggregate function from earlier into the lists for later usage, along with creating the titles for the conditions.

umsddf<-data.frame(Mean=c(unlist(mlist)), \n StDev=c(unlist(sdlist)), \n row.names=c(unlist(rnlist)) \n ) \n #Put together data frame using the unlist and concatenate functions to prevent issues \n \n print(umsddf) \n }

Then the data frame creation features the last main change, which is the usage of the lists I having been adding the variables to the entire time, by using the unlist and concatenate functions to do so. Admittedly, why this works but not just putting the list in does not is unknown to be, my best guess being the way lists are separated is different from how the data frame needs them to be. Regardless, it functions and so it can produce the output.

Mean: Total: 8.21 | Cond.  1 79.81 | Cond.  2 77.14 | Cond.  3 77.44 | Cond.  4 78.88 | StDev: Total: 11.244 | Cond. 1: 9.605 | Cond. 2: 9.456 | Cond. 3: 12.076 | Cond. 4: 13.776

If you want to see the whole code for any or all of these programs, as well their outputs in both csv and xlsx style, they can be found in the link below. So, that is all I want to show off with regards to the programs I have made. My plan for what I want to work on next when it comes to these programs will be to work on correlations and graphs in R.

Python Codes and Outputs
R Codes and Excel Files
Back to the blog