I wasn’t satisfied with just saying “check out what Martin did with Gource”, so I had a bit more of a look at what I could do. Firstly, I downloaded the UTS MDSI Slack logs, and worked out how to extract the information I needed. I did a bit of transformation, then exported the file in the appropriate format (code below the video).
I then played around with a few of the settings in Gource to make sure that it doesn’t run too slowly, zooms dynamically, doesn’t get too cluttered, etc, before saving it to file. The command for this was:
|
1 |
gource --camera-mode track --seconds-per-day 1 --auto-skip-seconds 0.1 --file-idle-time 30 --title "UTS MDSI Slack" --hide filenames -1280x720 -o - gource.log | ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i - -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 -bf 0 gource_slack.mp4 |
And now the video!
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
library(jsonlite) channels <- list.dirs("/Users/perrystephenson/data/slack") channels <- channels[-grep("slack$", channels)] channel_data <- vector(mode = "list", length = length(channels)) for (channel in 1:length(channels)) { channel_name <- gsub("/.*/", "", channels[channel]) files <- list.files(channels[channel], full.names = T) file_data <- vector(mode = "list", length = length(files)) for (file in 1:length(files)) { dat <- read_json(files[file]) day_data <- vector(mode = "list", length = length(dat)) for (i in 1:length(dat)) { excluded_subtypes <- c( "bot_message" ) if (!is.null(dat[[i]]$subtype)) { if (dat[[i]]$subtype %in% excluded_subtypes) { break } } # Timestamp timestamp <- dat[[i]]$ts # User if (is.null(dat[[i]]$user)) { username <- dat[[i]]$comment$user } else { username <- dat[[i]]$user } # Path dat[[i]]$text <- gsub("/", " ", dat[[i]]$text) file = paste(channel_name, dat[[i]]$text, sep = "/") # Put it all together day_data[[i]] <- data.frame( timestamp, username, type = "A", file ) } file_data[[file]] <- do.call("rbind", day_data) } channel_data[[channel]] <- do.call("rbind", file_data) } output <- do.call("rbind", channel_data) row.names(output) <- 1:nrow(output) output$file <- gsub("\\|", "-", output$file) output$timestamp <- as.integer(as.character(output$timestamp)) output$username <- as.character(output$username) output <- output[order(output$timestamp),] # Translate usernames dat <- read_json("/Users/perrystephenson/data/slack/users.json") users <- vector(mode = "list", length = length(dat)) for (i in 1:length(dat)) { if (!dat[[i]]$is_bot & !dat[[i]]$deleted) { users[[i]] <- data.frame( id = dat[[i]]$id, name = dat[[i]]$name, real_name = dat[[i]]$real_name ) } } users <- do.call("rbind", users) users$id <- as.character(users$id) users$name <- as.character(users$name) users$real_name <- as.character(users$real_name) row.names(users) <- 1:nrow(users) usernames <- users$name names(usernames) <- users$id output$username <- usernames[output$username] write.table(output, "~/data/gource.log", quote=F, sep="|", row.names = F,col.names = F) |
Hectic! Is there a way to highlight a specific user or community/channel so you can see their path in amongst it all.
For example exploring how involved/influential was user/node x?
I’m pretty sure you can, but you need to do it at “compile time”, because it’s generating video rather than an interactive visualisation.
Now I am definitely sure you can. Here is a video following you through the same visualisation:
https://vimeo.com/212428927
Well done sir! Very fancy 🙂
Trying to think of a way to express a logical, analytical, and well thought out response to this.
I cant. It just LOOKS SO COOL!!!
Anyway, I think I’m getting the message regarding what is happening as the “people” drift away and then are pulled back in as they post on a channel. But what is happening when a node spins and starts to “unravel”?
Ah, that is just when posts “expire”, which I’ve defined to be 30 seconds I think, which equates to about a month. I just had to do this to stop the clusters getting too large!