Ecoli_metadata.csv) as an example
We are studying a population of Escherichia coli (designated Ara-3), which were propagated for more than 40,000 generations in a glucose-limited minimal medium. This medium was supplemented with citrate which E. coli cannot metabolize in the aerobic conditions of the experiment. Sequencing of the populations at regular time points reveals that spontaneous citrate-using mutants (Cit+) appeared at around 31,000 generations. This metadata describes information on the Ara-3 clones and the columns represent:
|generation||generation when sample frozen|
|clade||based on parsimony-based tree|
|cit||citrate-using mutant status|
|run||Sequence read archive sample ID|
|genome_size||size in Mbp (made up data for this lesson)|
The metadata file required for this lesson can be downloaded by clicking on this link and should also be in the
~/dc_sample_data/r-genomics directory if you are using the connected to workshop genomics cloud machine.
getwd(). (This should be
~/dc_sample_data/r-genomicsin the workshop genomics cloud machine.
You are now ready to load the data. We are going to use the R function
read.csv() to load the data file into memory (as a
metadata <- read.csv('data/Ecoli_metadata.csv')
This statement doesn't produce any output because assignment doesn't display anything. If we want to check that our data has been loaded, we can print the variable's value:
Alternatively, wrapping an assignment in parentheses will perform the assignment and display it at the same time.
(metadata <- read.csv('data/Ecoli_metadata.csv'))
Wow... that was a lot of output. At least it means the data loaded properly. Let's check the top (the first 6 lines) of this
data.frame using the function
We've just done two very useful things. 1. We've read our data in to R, so now we can work with it in R 2. We've created a data frame (with the read.csv command) the standard way R works with data.
data.frame is the de facto data structure for most tabular data and what we use for statistics and plotting.
data.frame is a collection of vectors of identical lengths. Each vector represents a column, and each vector can be of a different data type (e.g., characters, integers, factors). The
str() function is useful to inspect the data types of the columns.
data.frame can be created by the functions
read.table(), in other words, when importing spreadsheets from your hard drive (or the web).
data.frame converts (= coerces) columns that contain characters (i.e., text) into the
factor data type. Depending on what you want to do with the data, you may want to keep these columns as
character. To do so,
read.table() have an argument called
stringsAsFactors which can be set to
Let's now check the structure of this
data.frame in more details with the function
We already saw how the functions
str() can be useful to check the content and the structure of a
data.frame. Here is a non-exhaustive list of functions to get a sense of the content/structure of the data.
dim()- returns a vector with the number of rows in the first element, and the number of columns as the second element (the dimensions of the object)
nrow()- returns the number of rows
ncol()- returns the number of columns
head()- shows the first 6 rows
tail()- shows the last 6 rows
names()- returns the column names (synonym of
rownames()- returns the row names
str()- structure of the object and information about the class, length and content of each column
summary()- summary statistics for each column
Note: most of these functions are "generic", they can be used on other types of objects besides
Based on the give table of functions to asses data structure, can you answer the following questions?
As you can see, many of the columns in our data frame are of a special class called
factor. Before we learn more about the
data.frame class, we are going to talk about factors. They are very useful but not necessarily intuitive, and therefore require some attention.
Factors are used to represent categorical data. Factors can be ordered or unordered and are an important class for statistical analysis and for plotting.
Factors are stored as integers, and have labels associated with these unique integers. While factors look (and often behave) like character vectors, they are actually integers under the hood, and you need to be careful when treating them like strings.
In the data frame we just imported, let's do
We can see the names of the multiple columns. And, we see that
some say things like
Factor w/ 30 levels
When we read in a file, any column that contains text is automatically assumed to be a factor. Once created, factors can only contain a pre-defined set values, known as levels. By default, R always sorts levels in alphabetical order.
For instance, we see that
cit is a Factor w/ 3 levels,
You can check this by using the function
levels(), and check the
number of levels using
Sometimes, the order of the factors does not matter, other times you might want to specify the order because it is meaningful (e.g., "low", "medium", "high") or it is required by particular type of analysis. Additionally, specifying the order of the levels allows to compare levels:
expression <- factor(c("low", "high", "medium", "high", "low", "medium", "high")) levels(expression) expression <- factor(expression, levels=c("low", "medium", "high")) levels(expression) min(expression) ## doesn't work expression <- factor(expression, levels=c("low", "medium", "high"), ordered=TRUE) levels(expression) min(expression) ## works!
In R's memory, these factors are represented by numbers (1, 2, 3). They are better than using simple integer labels because factors are self describing:
"high"" is more descriptive than
3. Which is low? You wouldn't be able to tell with just integer data. Factors have this information built in. It is particularly helpful when there are many levels (like the species in our example data set).
If you need to convert a factor to a character vector, simply use
Converting a factor to a numeric vector is however a little trickier, and you have to go via a character vector. Compare:
f <- factor(c(1, 5, 10, 2)) as.numeric(f) ## wrong! and there is no warning... as.numeric(as.character(f)) ## works... as.numeric(levels(f))[f] ## The recommended way.
table() tabulates observations and can be used to create bar plots quickly. For instance:
## Question: How can you recreate this plot but by having "control" ## being listed last instead of first? exprmt <- factor(c("treat1", "treat2", "treat1", "treat3", "treat1", "control", "treat1", "treat2", "treat3")) table(exprmt) barplot(table(exprmt))
exprmt <- factor(exprmt, levels=c("treat1", "treat2", "treat3", "control")) barplot(table(exprmt))