# Using Embeddings in R

August 2019 · 3 minute read

There exist different file formats to store distributed vector or word representations also known as embeddings. However, one of the most convenient ways is to use the text format used by the original word2vec implementation. In this format, each row starts with a label (an item of the vocabulary) followed by the vector components. Furthermore, each field is separated by an ordinary space. The following function processes such *.vec-files and can be used to load them directly into R:

# create new environment
embeddings_env <- new.env(hash = TRUE, parent = emptyenv())

# this function is used to convert vectors to unit vectors
# by dividing their components by vector length
normalize_vector <- function(a){
a/sqrt(sum(a**2))
}

# iterate through the whole file line by line
for (i in 1:length(lines)) {
line <- lines[[i]]
values <- strsplit(line, " ")[[1]]
label <- values[[1]]
embeddings_env[[label]] <- normalize_vector(as.double(values[-1]))
}

embeddings_env
}

For example, we trained a model on music data. Specifically, similarity between songs is measured by the co-occurrence on playlists. Therefore, all “labels” are in this case ISRC song IDs. Using our function, the embeddings can be imported by running emb <- load_embedding("example.vec"). This will create a new environment and store all vectors in it. The environment can be accessed by the variable emb. Sometimes you want to list all items in your vocabulary. Assuming your embedding is stored in a variable named emb it is as easy as calling ls.str(emb).

By using the $-operator one can also extract the vector representation for a given label. In the music example, you get the vector for the song with the id NLB390800100 (aka Punk Rock Song by Bad Religion) by running emb$NLB390800100.

### Calculating cosine similarity

The similarity between two items is commonly measured by the angle between their vectors. Because, we normalized all vectors to unit-vectors while importing, the cosine is given by the dot-product. For example, we can calculate the similarity between NLB390800100 and USUM71213718 like this: sum(emb$NLB390800100*emb$USUM71213718). Alternatively, one can shorten this expression by using a single matrix multiplication (e.g., emb$NLB390800100 %*% emb$USUM71213718).

### Get most similar elements

We can use the pairwise similarity to find the n most similar items given a reference item.

cosine_similarity <- function(a,b){
# assuming unit vectors
# the cosine is just the dot-product
a %*% b
}

most_similar <- function(embeddings, ref_item, n_top = 10){
# calculate cos similarity to ref_item for all elements
cos_sims <- eapply(embeddings, cosine_similarity, b = ref_item)

# only look at cos values smaller than 1
# this will ignore the same element
cos_sims <- cos_sims[cos_sims < 1]