Filip Agneessens

John Skvoretz

Social Networks

download the paper directly

preprint with the exact same text

Abstract

Ties often have a strength naturally associated with them that differentiate them from each other. Tie strength has been operationalized as weights. A few network measures have been proposed for weighted networks, including three common measures of node centrality: degree, closeness, and betweenness. However, these generalizations have solely focused on tie weights, and not on the number of ties, which was the central component of the original measures. This paper proposes generalizations that combine both these aspects. We illustrate the benefits of this approach by applying one of them to Freeman’s EIES dataset.

Motivation

Ego networks of Phipps Arabie (A), John Boyd (B), and Maureen Hallinan (C) from Freeman's third EIES network. The width of a tie corresponds to the number of messages sent from the focal node to their contacts. Adopted from the paper.

The three measures have already been generalised to weighted networks. Barrat et al. (2004) generalised degree to weighted networks by taking the sum of weights instead of the number ties, while Newman (2001) and Brandes (2001) utilised Dijkstra’s (1959) algorithm of shortest paths for generalising closeness and betweenness to weighted networks, respectiviely. Dijkstra’s algorithm defined the length of paths as the sum of cost (e.g., time in GPS calculations), which is generally only defined as the sum of the inversed tie weights. All these generalisations fail to take into account the main feature of the original measures formalised by Freeman (1978): the number of ties.

This limitation is highlighted for degree centrality by the three ego networks from Freeman’s third EIES network. The three nodes have roughly sent the same amount of messages; however, to a quite different number of others. If Freeman’s (1978) original measure was applied, the centrality score of the node in panel A is almost five times as high as the node in panel C attains. However, when using Barrat et al.’s generalisation, they get roughly the same score.

This articles proposes a new generation of node centrality measures for weighted networks. The second generation of measures takes into consideration both the weight of ties and the number of ties. The relative importance of these two aspects are controlled by a tuning parameter.

Want to test it with your data?

The degree_w, closeness_w, and betweenness_w-functions in tnet allows you to calculate the binary, weighted, and the measures that combine these two aspects on your own dataset.

For example, to calculate second generation node centrality measures (alpha = 0.5) on the sample network above, you can run the code below in R. The degree function easily calculates the binary and first generation measures as well; however, this is not the case for the closeness and betweenness-functions. If you would like the binary version, you can either use the dichotomise function or set alpha=0. If you would like the first generation weighted measures, you can set alpha=1 (default value).

# Load tnet
library(tnet)

# Load network
net <- cbind(
i=c(1,1,2,2,2,2,3,3,4,5,5,6),
j=c(2,3,1,3,4,5,1,2,2,2,6,5),
w=c(4,2,4,4,1,2,2,4,1,2,1,1))

# Calculate degree centrality
degree_w(net, measure=c("degree", "output", "alpha"), alpha=0.5)

# Calculate closeness centrality
closeness_w(net, alpha=0.5)

# Calculate betweenness centrality
betweenness_w(net, alpha=0.5)

To test it on Freeman’s third EIES network from the datasets page and recreate Table 3 of the paper, you can do the following:

# Load tnet
library(tnet)

# Load network
data(Freemans.EIES)
net <- Freemans.EIES.net.3.n32

# Calculate measures
tmp <- data.frame(
  Freemans.EIES.node.Name.n32,
  degree_w(net, measure=c("degree", "output", "alpha"), alpha=0.5),
  degree_w(net, measure="alpha", alpha=1.5)[,"alpha"], stringsAsFactors=FALSE)
dimnames(tmp )[[2]] <- c("name", "node", "a00", "a10", "a05", "a15")
tmp <- tmp[,c("name","a00","a05","a10","a15")]

# Merge names and order table
out <- data.frame(
  seq.int(nrow(tmp)),
  tmp[order(-tmp[,"a00"], -tmp[,"a10"]),c("name", "a00")],
  tmp[order(-tmp[,"a05"], -tmp[,"a10"]),c("name", "a05")],
  tmp[order(-tmp[,"a10"], -tmp[,"a10"]),c("name", "a10")],
  tmp[order(-tmp[,"a15"], -tmp[,"a10"]),c("name", "a15")])
dimnames(out)[[2]] <- c("Rank",
  "a00.name","a00",
  "a05.name","a05",
  "a10.name","a10",
  "a15.name","a15")

# Display table
out

References

Brandes, U., 2001. A Faster Algorithm for Betweenness Centrality. Journal of Mathematical Sociology 25, 163-177.

Dijkstra, E. W., 1959. A note on two problems in connexion with graphs. Numerische Mathematik 1, 269-271.

Freeman, L. C., 1978. Centrality in social networks: Conceptual clarification. Social Networks 1, 215-239.

Newman, M. E. J., 2001. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E 64, 016132.

Opsahl, T., Agneessens, F., Skvoretz, J. (2010). Node centrality in weighted networks: Generalizing degree and shortest paths. Social Networks 32, 245-251.

This network gives a concrete example of the closeness measure. The distance between node G and node H is infinite as a direct or indirect path does not exist between them (i.e., they belong to separate components). As long as at least one node is unreachable by the others, the sum of distances to all other nodes is infinite. As a consequence, researchers have limited the closeness measure to the largest component of nodes (i.e., measured intra-component). The distance matrix for the nodes in the sample network is:

Although the intra-component closeness scores are not infinite for all the nodes in the network, it would be inaccurate to use them as a closeness measure. This is due to the fact that the sum of distances would contain different number of paths (e.g., there are two distance from node H to other nodes in its component, while there are six distances from node G to other nodes in its component). In fact, nodes in smaller components would generally be seen as being closer to others than nodes in larger components. Thus, researchers has focused solely on the largest component. However, this leads to a number of methodological issues, including sample selection.

To develop this measure, I went back to the original equation:

where is the focal node, is another node in the network, and is the shortest distance between these two nodes. In this equation, the distances are inversed after they have been summed, and when summing an infinite number, the outcome is infinite. To overcome this issue while staying consistent with the existing measure of closeness, I took advantage of the fact that the limit of a number divided by infinity is zero. Although infinity is not an exact number, the inverse of a very high number is very close to 0. In fact, 0 is returned if you enter 1/Inf in the statistical programme R. By taking advantage of this feature, it is possible to rewrite the closeness equation as the sum of inversed distances to all other nodes instead of the inversed of the sum of distances to all other nodes. The equation would then be:

To exemplify this change, for the example network above, the inversed distances and closeness scores are:

As can be seen from this table, a closeness score is attained for all nodes taking into consideration an equal number of distances for each node irrespective of the size of the nodes’ component. Moreover, nodes belonging to a larger component generally attains a higher score. This is deliberate as these nodes can reach a greater number of others than nodes in smaller components. The normalized scores are bound between 0 and 1. It is 0 if a node is an isolate, and 1 if a node is directly connected all others.

This measure can easily be extended to weighted networks by introducing Dijkstra’s (1959) algorithm as proposed in Average shortest distance in weighted networks.

References

Dijkstra, E. W., 1959. A note on two problems in connexion with graphs. Numerische Mathematik 1, 269-271.

Freeman, L. C., 1978. Centrality in social networks: Conceptual clarification. Social Networks 1, 215-239.

Opsahl, T., Agneessens, F., Skvoretz, J. (2010). Node centrality in weighted networks: Generalizing degree and shortest paths. Social Networks 32, 245-251.

Wasserman, S., Faust, K., 1994. Social Network Analysis: Methods and Applications. Cambridge University Press, New York, NY.

What to try it with your data?

Below is the code to calculate the closeness measure on a binary version of Freeman’s third EIES network. From tnet version 2.7, this measure will be included in the closeness-function.

# Load tnet 2.7 or newer
library(tnet)

# Load network
# Node K is assigned node id 8 instead of 10 as isolates at the end of id sequences are not recorded in edgelists
net <- cbind(
  i=c(1,1,2,2,2,3,3,3,4,4,4,5,5,6,6,7,9,10,10,11),
  j=c(2,3,1,3,5,1,2,4,3,6,7,2,6,4,5,4,10,9,11,10),
  w=c(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1))

# Calculate measures
closeness_w(net, gconly=FALSE)

In a similar vein as the global clustering coefficient that I proposed in Clustering in two-mode networks, the local clustering coefficient can be redefined for two-mode networks. Originally, Watts and Strogatz (1998) defined the local clustering coefficient for a focal node as the fraction of present ties among a node’s neighbors over the possible number of ties between them. It can be formalized for a focal node, as follows:

where is the number of 2-paths centered on a node, and is the number of these that are closed. While the global clustering coefficient is an aggregation of all 2-paths, the local one can be seen as simply an intermediary level of aggregation as it can be conceptualized in terms of 2-paths.

When applying the traditional local clustering coefficient to the projection of two-mode network, cliques among nodes connected to common nodes in the two-mode network are created. These cliques contain a high number of triangles. This has an impact on measures that rely on ego network density, such as the clustering coefficients and structural holes measures (Burt, 1992). The average of local clustering coefficients is over-estimated for these networks as projections of random two-mode networks contain an above random clustering coefficient. Therefore, a new measure that does not over-estimate the level is needed.

Given the extention of 2-paths in one-mode networks to 4-paths in two-mode networks for the global clustering coefficient, the denominator and numerator of the local clustering coefficient can also be redefined in terms of 4-paths. While the original local coefficient was based on 2-paths centered on the focal node, this can be extended to 4-paths centered on a focal node in two-mode networks. This would imply that the first and last nodes of the path are of the same mode as the focal node. Formally, I propose:

where is the number of 4-paths with ego as the middle node, and is the subset of these in which the first and the last nodes of the path share a common node that is not part of the 4-path.

This coefficient has similar properties as the local clustering coefficient. First, for each node, the coefficient varies between 0 and 1 as the numerator and denominator are positive numbers, and the numerator is a subset of the denominator. Second, all 4-paths are closed in a fully connected network, and therefore, the coefficient is equal to 1. Third, if ties are randomly placed in the network, the expected value of the local clustering coefficient is the same as the one for the global coefficient, where is the density of the network.

Empirical test

To empirically test the proposed local clustering coefficient for two-mode networks, I have also used the Davis Southern Women dataset as this dataset has a limited number of nodes. The table below shows the local clustering coefficients attained from the two-mode network and projected one-mode network as well as the two-mode and one-mode degree scores (i.e., the number of events attended and the number of other women attending the same events, respectively).

The two-mode and one-mode degree scores and the traditional and proposed local clustering coefficients (LCC) of the women in Davis’ (1940) Southern Women dataset. The randomly expected one-mode clustering coefficient is 0.9085, while the one for two-mode networks is 0.7978.

There are a number of observations. First, for all the nodes that did not have the maximum value, the two-mode coefficient is smaller than the coefficient attained on the projected network. This feature is not given as multiple 4-paths might exist among three primary nodes, and therefore, the two-mode coefficient might be higher than the one attained on projected one-mode network. It gives, however, an indication of the bias that is created by three or more primary nodes are connected to a common node. Second, the reduction difference between the two coefficients is greater for the women attending fewer events (pair-wise correlation between the number of events and the difference is -0.69, with a -value of 0.001). This might suggest that the bias is greatest for nodes that attend few events. This is not unexpected as a woman attending a single event with at least two others would automatically attain a coefficient of 1 in the binary network.

To further highlight some of the features of the redefined local clustering coefficient, Flora and the network around her up to three steps is shown below. In a one-mode projection, all the possible ties among Flora’s contacts are present. This is due to the fact that eleven out of the twelve contacts attended event 9. The twelfth contact that did not attend event 9, Helen, is connected to all others connect through other events. The redefined clustering coefficient is less than 1 for Flora. This is because event 9 and 11 are not used to form closing ties among the women attending them (i.e., close 4-paths). More specifically, 4-paths exist from the nodes attached to event 11 to the nodes connected to event 9 (excluding themselves). In total, there are 31 4-paths, out of which 18 are closed by the event 6, 7, 8, and 10.

Flora’s local network up to three steps. Only non-redudant ties are shown between the second and third steps.

References

Burt, R.S., 1992. Structural Holes. Harvard University Press, Cambridge, MA.
Davis, A., Gardner, B. B., Gardner, M. R., 1941. Deep South. University of Chicago Press, Chicago, IL.
Watts, D.J., Strogatz, S.H., 1998. Collective dynamics of small-world networks. Nature 393, 440-442.

What to try it with your data

The redefined local clustering coefficient is implemented in tnet as clustering_tm_local. Below is the code for analysing a sample network and Davis’ (1940) Southern Women dataset is shown.

# Load tnet
library(tnet)

# Load networks
net <- cbind(
 i=c(1,1,2,2,2,3,3,4,5,5,6),
 p=c(1,2,1,3,4,2,3,4,3,5,5),
 w=c(3,5,6,1,2,6,2,1,3,1,2))

# Obtain the binary local clustering coefficients of the nodes in the sample network
clustering_tm_local(net[,1:2])

# Obtain the weighted local clustering coefficients of the nodes in the sample network
clustering_tm_local(net)

# Obtain the binary local clustering coefficients of the women in Davis' (1940) Southern Women dataset
data("Davis.Southern.women")
clustering_tm_local(Davis.Southern.women.2mode)

The output from the binary and weighted analyses of the sample network is:

 node  lc
    1 1.0
    2 0.2
    3 0.5
    4 NaN
    5 0.0
    6 NaN

 node  lc     lc.am     lc.gm     lc.ma     lc.mi
    1 1.0 1.0000000 1.0000000 1.0000000 1.0000000
    2 0.2 0.2400000 0.2313222 0.2608696 0.2000000
    3 0.5 0.4666667 0.4317651 0.5000000 0.3333333
    4 NaN       NaN       NaN       NaN       NaN
    5 0.0 0.0000000 0.0000000 0.0000000 0.0000000
    6 NaN       NaN       NaN       NaN       NaN

Ph.D. thesis

the Dataset-page

Patterns and Dynamics of Users’ Behaviour and Interaction: Network Analysis of an Online Community

Prominence and control: The weighted rich-club effect

Clustering in weighted networks

Citation: Opsahl, T., Panzarasa, P., 2009. Clustering in weighted networks. Social Networks 31 (2), 155-163, doi: 10.1016/j.socnet.2009.02.002

A sample page from the online social network

Patterns and Dynamics of Users’ Behaviour and Interaction: Network Analysis of an Online Community

“we measured users’ average out-strength (instrength) as the average number of messages sent to (received from) others (Opsahl, Colizza, Panzarasa,&Ramasco, 2008). We expected hubs to be weakly connected to others, based on the conjecture that all users are homogeneously limited by the same constraints of resources and time. In this case, having more contacts should reduce the amount of resources and time spent on each of them (Burt, 1992). We were surprised to find a positive and significant (p<0.001) Pearson’s pairwise correlation coefficient between average out-strength (in-strength) and out-degree (in-degree) of 0.28 (0.44). This signals that hubs spend more time and resources with each of their contacts than the less connected users." (excerpt from page 919).

The heterogeneity in average tie weight for users with different levels of gregariousness might indicate that node degree and node strength are not correlated. This post aims to test this for the online social network used in the paper and compare degree and strength distributions.

Given that this is a directed network, each analysis is conducted twice – once for outgoing ties and once for incoming ties. The simplest way to test the association between two variables is to calculate the Pearson pair-wise correlation coefficient . This coefficient tests the linear relationship between two variables, and ranges from -1 to 1. If it is equal to 1, then there is perfect correlation between the two-variables, whereas if it is -1, the two variables are opposites of each other. A value of 0 is attained if there is no linear relationship between the two variables. For out-degree and out-strength, the coefficient is 0.90, and for in-degree and in-strength, the coefficient is 0.89. This indicates that degree and strength is highly correlated with each other (Cohen, 1988).

Since high correlation coefficients were found, it might be interesting to plot the relationships to ensure that extreme values are not distorting the coefficient. The relationships between the two types of degree and strength are:

As it is possible to see from the above plots, there are a number of nodes with extremely high values of degree and strength. However, there are clear trajectories at low values of degree and strength, which might indicate that the outliers are not distorting the correlation coefficients. The fact that there are nodes with extremely high values of degree is not surprising given that power-law degree distributions with exponents of 0.89 and 1.005 were found in the paper:

Given the similarity between degree and strength, it would be interesting to test whether the strength distributions also follow a power-law distribution, and if so, if the exponent is similar to the ones for the degree-distributions:

The exponents of the strength distributions are 0.87 and 1.004. Although I expected some similarity between the degree distributions’ exponents (0.89 and 1.005) and the strength distributions’ exponents, the numerical similarity is striking.

References

Burt, R. S., 1992. Structural Holes: The Social Structure of Competition. Harvard University Press, Cambridge, MA.

Cohen, J., 1988. Statistical power analysis for the behavioral sciences (2nd edition). Hillsdale, NJ: Erlbaum.

Opsahl, T., Colizza, V., Panzarasa, P., Ramasco, J. J., 2008. Prominence and control: The weighted rich-club effect. Physical Review Letters 101 (168702). arXiv:0804.0417.

Panzarasa, P., Opsahl, T., Carley, K.M., 2009. Patterns and dynamics of users’ behavior and interaction: Network analysis of an online community. Journal of the American Society for Information Science and Technology 60 (5), 911-932, doi: 10.1002/asi.21015

What to try it with your data?

Below is the code to calculate the numbers and create the diagrams used in this post. If you also would like to calculate the power-law with exponential cut-off, then you should remove the # on line 41.

# Load tnet
library(tnet)

# Load network
data(OnlineSocialNetwork.n1899)

`script` <-
function(net){
  output <- list()
  # Calculate out/in-degree/strength
  k <- cbind(degree_w(net), degree_w(net, type="in"))
  dimnames(k)[[2]] <- c("node","ko","so","node2","ki","si")
  if(sum(k[,"node"] == k[,"node2"])!=nrow(k))
    stop("Node ids does not match")
  k <- k[,c("node","ko","ki","so","si")]
  output[[1]] <- k

  # Get pair-wise correlation coefficients
  corro <- cor.test(k[,"ko"], k[,"so"])
  corri <- cor.test(k[,"ki"], k[,"si"])
  cat(paste("Pair-wise correlation between degree and strength:\n Out: ", corro$estimate, " (p-value: ", corro$p.value, ")\n In:  ", corri$estimate, " (p-value: ", corri$p.value, ")\n Note: If p-value equal 0, p-value is less than 2.2e-16\n", sep=""))
  output[[2]] <- corro
  output[[3]] <- corri

  # Degree distributions
  cat("Degree distributions\n")
  looprange <- c("ko","so","ki","si")
  for(j in 1:length(looprange)) {
    i <- looprange[j]
    tmp <- table(k[,i])
    tmp <- tmp[which(rownames(tmp)!="0")]
    tmp <- tmp/(sum(tmp))
    tmp <- as.data.frame(cbind(k=as.numeric(rownames(tmp)), pk=tmp))
    plaw <- nls(pk ~ C*k^(-t), data=tmp, start=list(C=1, t=1))
    plaweco <- nls(pk ~ C*k^(-t)*exp(-k/K), data=tmp, start=list(C=1, t=1, K=30))
    cat(switch(i,
      "ko" = " Out-degree",
      "so" = " Out-strength",
      "ki" = " In-degree",
      "si" = " In-strength"))
    cat(paste("\n  Powerlaw:  pk =", plaw$call$formula[3], "\n   Coefficients:\n    Con =", coef(plaw)["C"], "\n    tau =", coef(plaw)["t"]))
    # cat(paste("\n  Powerlaw with exponential cut-off: pk ", plaweco$call$formula[3], "\n   Coefficients:\n    Con =", coef(plaweco)["C"], "\n    tau =", coef(plaweco)["t"], "\n    cut =", coef(plaweco)["K"]))
    cat("\n")
    output[[(length(output)+1)]] <- tmp
    output[[(length(output)+1)]] <- plaw
    output[[(length(output)+1)]] <- plaweco
  }
  cat(" Note: These regressions in the article were performed in Stata 9\n The value of the cut-off parameter varies slightly between R and Stata\n")
  return(output)
}
output <- script(OnlineSocialNetwork.n1899.net)
k <- output[[1]]
plot(k[,"ko"], k[,"so"], main="Outgoing ties", xlab="out-degree", ylab="out-strength")
plot(k[,"ki"], k[,"si"], main="Incoming ties", xlab="in-degree",  ylab="in-strength" )

plot(output[[4]][,1], output[[4]][,2], main="Out-degree distribution", xlab="out-degree", ylab="p(out-degree)", log="xy")
lines(output[[4]][,1], fitted(output[[5]]))

plot(output[[7]][,1], output[[7]][,2], main="Out-strength distribution", xlab="out-strength", ylab="p(out-strength)", log="xy")
lines(output[[7]][,1], fitted(output[[8]]))

plot(output[[10]][,1], output[[10]][,2], main="In-degree distribution", xlab="in-degree", ylab="p(in-degree)", log="xy")
lines(output[[10]][,1], fitted(output[[11]]))

plot(output[[13]][,1], output[[13]][,2], main="In-strength distribution", xlab="in-strength", ylab="p(in-strength)", log="xy")
lines(output[[13]][,1], fitted(output[[14]]))

I would like to acknowledge Vittoria Colizza in helping to develop the idea behind this post.

Introduction

Networks are representations of systems in which the elements (or nodes) are connected by ties (Wasserman and Faust, 1994). Most networks are defined as one-mode networks with one set of nodes that are similar to each other. However, many network dataset are in fact two-mode networks (also known as affiliation or bipartite networks). These networks are a particular kind of networks with two different sets of nodes (e.g., employees and projects), and ties are only established between nodes belonging to different sets (e.g., employees are linked to a project if they have worked on it). In most two-mode networks, the nodes of one node set are considered more responsible for tie creation (primary node set) than the other (secondary node set). For example, employees decide whether or not to collaborate on a project.

One of the first two-mode datasets to be analysed was the Davis Southern Club Women dataset (Davis et al., 1941), which recorded the attendance of a group of women (node set 1) to a series of events (set 2). A woman would be linked to an event if she attended it. Another category of two-mode networks that has become popular in recent years is scientific collaboration networks (e.g., Newman, 2001). In these networks, the two sets of nodes are scientists (node set 1) and papers (node set 2), and a tie is established if a scientist have authored a paper.

Two-mode networks are rarely analysed without transforming them. A likely cause of this is that most network measures are defined for one-mode network, and few of them have been redefined for two-mode networks. Therefore, researchers have constructed one-mode networks from the two-mode networks. This is often done using a method referred to as projection. This method operates by selecting one of the two node sets and linking two nodes from that set if they were connected to a common node of the other set. For example, interlocking directorates can be studied in two ways from a network perspective (Mizruchi, 1996; Zajaz, 1988). In this kind of networks, the two node sets are directors and corporate boards, and ties represent the affiliation of directors with boards. On the one hand, it is possible to study the interdependence among companies. Levine (1979) studied how companies were connected by having the same individual on both their boards of directors. By being connected, communication, knowledge transfer, and coordination are facilitated between companies (Zajaz, 1988). On the other, it is possible to study the interpersonal relationships among individuals that are directors on a set of boards of directors. In this case, individuals can be linked by being members of the same boards (Opsahl and Seierstad, 2009).

Projection of two-mode networks creates a number of modelling issues. First, each tie in a regular one-mode network is created separately; however, this is not the case in projected two-mode networks. For example, while a standard phone call creates a communication tie from one person to another, a director forms ties with all the other directors on a board when she or he joins that board. This has direct implications for models that utilize random networks to create benchmark values (e.g., Opsahl et al., 2008) and when comparing values found in an observed network with those found in corresponding random networks. Ties in random networks are assumed to be independent of each other. (Footnote 1) Although this is neither the case in observed one-mode nor two-mode networks, in projected two-mode networks, ties are not even formed separately. Thus, they are less comparable to the observed network than to genuine one-mode networks. Second, depending on the degree distribution of the non-selected node set, a projected one-mode network tends to have more and larger fully-connected cliques than non-projected one-mode networks (Wasserman and Faust, 1994). These are produced when a group of nodes are connected to the same other node in the two-mode network (e.g., all the directors on a single board are all connected). This feature impacts a number of network measures, and in particular, the clustering coefficient (for a review, see Opsahl and Panzarasa, 2009). This measure is the fraction of 2-paths (i.e., three nodes connected by two ties) that are part of a triangle. Given that the cliques contain a number of triangles, this measures might be biased. To exemplify the cliques, and the many triangles, produced when projecting a two-mode network, Figure 1 shows the main component of the interpersonal network among Norwegian directors (Opsahl and Seierstad, 2009).

Figure 1: The network structure among directors (circles) who form part of the largest group of interconnected directors. Two directors are connected if they are members of the same board. The solid circles refer to women, whereas the hollow circles refer to men.

The rest of the paper is organised as follows. First, I will describe the binary clustering coefficient and its properties. Then, I will propose a new measure that detects triadic closure in two-mode networks while omitting triangles formed by three nodes solely being connected to the same other nodes (e.g., three scientists writing a paper). I will then suggest a method for integrating the generalization by Opsahl and Panzarasa (2009) of the clustering coefficient to weighted one-mode networks by offering a generalisation of the proposed coefficient to weighted two-mode networks. This will be followed by applications of the proposed coefficients to four two-mode networks from the domains of event attendance, scientific collaboration, interlocking directorates, and online communication. Finally, I will offer some concluding remarks.

Clustering coefficient for one-mode networks

A measure that has long received much attention in both theoretical and empirical research is concerned with the degree to which nodes tend to cluster together. Evidence suggests that in most real-world networks, and especially social networks, nodes tend to cluster into densely connected groups (Holland and Leinhardt, 1970; Opsahl and Panzarasa, 2009; Watts and Strogatz, 1998). Two main measures have been developed for testing this tendency: the local clustering coefficient (Watts and Strogatz, 1998) and the global clustering coefficient (Luce and Perry, 1949; Opsahl and Panzarasa, 2009). The global coefficient assesses the overall level of clustering in the network, whereas the local coefficient measures the density of ties among a node’s contacts. This paper is concerned with the former of these two measures. This coefficient is the fraction of triplets or 2-paths that are closed by the presence of a tie between the first and the third node. It is formally defined as:

where is the number of 2-paths, and is the number of these 2-paths that are closed by being part of a triangle. This coefficient varies between 0 and 1. It is equal to 0 if no triangles exist in the network, and equal to 1 if all 2-paths are closed. In a completely connected network, the coefficient would be 1 as all 2-paths would be closed. Moreover, in a classical random network, the expected value of the clustering coefficient is equal to the probability of a tie being formed (i.e., the density) as ties are independent of each other (Erdos and Rényi, 1959).

Clustering coefficient for two-mode networks

In two-mode networks, the denominator and numerator of the clustering coefficient can be redefined in terms of 4-paths and closed 4-paths, respectively. (Footnote 2) First, a 4-path is a path that starts at a node, goes via two nodes of the other node set and one node of the same node set and, and ends up at a node of the same node set. For example, in Figure 2a, a 4-path exists between node 1 and node 3 via node A, node 2, and node C. Second, a closed 4-path is a 4-path in which the two end nodes are connected to a common node (excluding the nodes that are already part of the path). For example, the mentioned 4-path between node 1 and node 3 is closed as these nodes are both connected to node B, which is not part of the path.

Figure 2: (a) A two-mode network where the shape and colour of nodes represent the set to which a node belongs, and (b) the one-mode projection of the two-mode network in panel a. The round blue nodes are the primary nodes in this projection. The clustering coefficient of the two-mode network (panel a) is 0.3, while the clustering coefficient of the one-mode projection (panel b) is 0.4615.

All 4-paths in a two-mode network are 2-paths in a one-mode projection of the network; however, not all 2-paths in a one-mode projection are created from 4-paths. In fact, 2-paths can also be created due to multiple nodes being connected to the same node. The 2-paths created by the latter mechanism would be excluded when only considering 4-paths in the two-mode structure. This feature is illustrated in Figure 2. In the first panel (a), there are ten 4-paths, three of which are closed. (Footnote 3) These 4-paths represent ten 2-paths in the one-mode projection (panel b). (Footnote 4) However, in the one-mode projection (panel b), there are an additional three 2-paths. These are created among node 2, node 3, and node 5 as these nodes are all connected to node C in the two-mode network.

Formally, the proposed coefficient can be defined as:

where is the number of 4-paths, and is the number of these 4-paths that are closed by being part of a 6-cycle (i.e., a loop of six ties with five nodes).

The proposed coefficient has a number of properties. First, it varies between 0 and 1 as the numerator and denominator are positive numbers, and the numerator is a subset of the denominator. Second, in a fully connected network, the coefficient is equal to 1 as all 4-paths are closed. Third, in an ensemble of random two-mode networks with a set number of nodes and ties, the average clustering coefficient is approximately , where is the density (i.e., for two-mode networks), is the number of primary nodes, and is the number of secondary nodes). (Footnotes 5 and 6)

Weighted networks

The one-mode clustering coefficient has been extended to weighted networks by assigning a value to each triplet or 2-path, (Opsahl and Panzarasa, 2009). This value is based on the weights of the two ties that compose the 2-path, e.g., the arithmetic mean, geometric mean, minimum, or maximum of the tie weights. The coefficient incorporates these values by being defined as the total value of closed 2-paths over the total value of all 2-paths. In addition to having the same parameters as the original coefficient, this coefficient produces the same outcome as the original one if all 2-paths have the same value (e.g., a binary network) or if the weights are randomly reshuffled in the network.

The coefficient proposed in this paper can be generalized to weighted two-mode networks, such as those created from online forums where the weights is the number of messages or characters posted to a topic. In a similar spirit as the generalization to weighted networks of the one-mode clustering coefficient, a 4-path-value could be defined. This valued should be constructed based on the four tie weights, and could be defined as the arithmetic mean, geometric mean, minimum, or maximum. For example, the 4-path from node 1 to node 4 (via nodes A, 2, and D) in Figure 3 could be assigned a value of 3, 2.45, 1, or 6, respectively. The clustering coefficients based on the four methods for the sample network shown in Figure 3 would be 0.4, 0.398, 0.385, and 0.383. The topology of the networks is identical to the binary two-mode network in Figure 2, which had a clustering coefficient of 0.3. The increase in the coefficient when tie weights are considered is a reflection of the fact that the closed 4-paths have relatively stronger ties than the open 4-paths. The various explanations given in Opsahl and Panzarasa (2009) for the differences between the methods for defining 2-paths values are also applicable in the case of 4-path values. Formally, the coefficient for weighted two-mode networks could be defined as:

Figure 3: A weighted two-mode network.

The generalized coefficient has a number of properties. In addition to having the same properties as the binary two-mode clustering coefficient, the generalized coefficient is equal to the binary one when all ties have the same value (i.e., binary). Moreover, the outcome of the generalised coefficient is approximately equal to the binary one if tie weights are randomly reshuffled in the network.

Empirical test

To illustrate the proposed coefficient, I will apply it to Davis Southern Club Women dataset (Davis et al., 1941), Newman’s (2001) scientific collaboration network, the interpersonal network among directors on Norwegian public limited company boards (Opsahl and Seierstad, 2009), and data collected from an online forum. First, Davis Southern Club Women was collected in the 1930s, and contains the attendance of 18 women to 14 events. This network is relatively dense as 91 percent of the possible ties are present in the one-mode projection, and a slightly higher clustering coefficient is found (0.9284). Thus, one would assume that a triadic closure effect drives tie formation. However, the observed two-mode clustering coefficient (0.77) is less than the randomly expected value (0.80). This suggests that a triadic closure effect is not at play, in fact, it suggests an opposite reverse effect.

Second, Newman’s (2001) collaboration network is based upon 22,016 co-authored papers published on the arXiv e-repository website between 1995 and 1999. In total, these papers have 16,726 authors listed. The one-mode projection of this network using the authors as the primary node set has often been used as an empirical example for a variety of network measures (e.g., Opsahl et al., 2008). In the projected one-mode network, a clustering coefficient of 0.3596 is obtained, while the randomly expected value is 0.0003. Due to the great difference between the observed coefficient and the expected one, it has been argued that there is a strong tendency towards clustering (Newman, 2001). However, this number includes many of the triangles that are form by construction as the average number of authors per paper is 2.66. (Footnote 7) The proposed clustering coefficient for two-mode networks can be applied in an effort to exclude these triangles. This coefficient is 0.2769, while the randomly expected value is 0.0006. Albeit a weaker effect than the one found in the projected one-mode network, this suggests that there is an actual triadic closure effect in this network.

Third, the interlocking directorate dataset is composed of the 367 Norwegian public limited company boards as of August 1, 2009 (Opsahl and Seierstad, 2009). Based on this dataset, it is possible to construct an interpersonal network among the 1,413 directors, and link two individuals if they sit on the same board. In this one-mode projection, the observed and randomly expected clustering coefficients are 0.6477 and 0.0035, respectively, giving a ratio between them of 185. However, in the two-mode network, the observed and randomly expected clustering coefficients are 0.0119 and 0.0041, respectively. As the ration between these two coefficients is only 2.93, a much weaker triadic closure effect in the network is suggested than assumed from the projected one-mode network.

Fourth, the online forum data was collected from an online community of students at University of California, Irvine, in 2004. This community consisted of 2,995 students who could create any group that they wanted, and post messages to any group that they were a member of. In total, 889 students posted 33,720 messages to 552 groups. On average, each student posted 4.76 messages or 480.19 characters to each group they actively participated in. A two-mode network can be constructed from this dataset by linking a student to a group if she or he posted to it. Unlike the other datasets, it is possible to create both binary and weighted to-mode networks from this one. As weights can be assigned to each tie based on either the number of messages or characters posted, I have created a binary and two weighted two-mode networks from this dataset. The various clustering coefficients obtained from these networks are listed in Table 1. (Footnote 8)

Network			2M	1M	1M CC	Random 1M CC	2M CC	Random 2M CC
Davis Southern Women Club	18	14	89	139	0.9284	0.9085	0.7719	0.7978
Scientific collaboration network	16,726	22,016	58,595	47,594	0.3596	0.0003	0.2769	0.0006
Interpersonal network among directors	1,413	367	1,734	3,414	0.6477	0.0035	0.0119	0.0041
Online forum (binary)	899	522	7,089	16,860	0.5049	0.1768	X	0.1119
Online forum (weighted, messages)	…	…	…	…	…	…	X	0.1119
Online forum (weighted, characters)	…	…	…	…	…	…	X	0.1119

Table 1: The various clustering coefficients (CC) obtained on the four empirical datasets. 2M refers to two-mode networks, and 1M to one-mode ones. The one-mode clustering coefficients are obtained on binary projections.

Concluding remarks

Two-mode networks are rarely analysed without projecting them onto one-mode networks as there are few methods for this purpose. However, by projecting two-mode networks, certain assumptions in the one-mode methods might be violated, such as the ability of each tie to be formed separately. Moreover, measures based on triangles or ties among nodes’ contacts might be biased. This is due to the fact that projected two-mode networks generally contain more and larger fully-connected cliques than regular one-mode networks. In particular, depending on the degree distribution of the non-projected nodes, the measure that assesses the overall level of clustering in a one-mode network, the clustering coefficient, could be biased. More specifically, if the non-projected nodes have a degree greater than 2, triangles will be automatically formed in the one-mode projection, which will increase the coefficient. Thus, there is a need to redefine one-mode measures for two-mode networks, and in particular, the ones that are directly affected by the increase in triangles.

This paper proposed such a redefinition for the clustering coefficient. It subtracts the level of triadic closure caused by groups of nodes that are connected to common nodes in the two-mode structure. More specifically, it excludes triplets that are formed due to this effect, and only counts the number of triplets that are formed based on independent dyadic interactions.

The proposed coefficient is not without its limitations. A major one is that one of the node sets must be considered to be responsible for tie generation, and designated as the primary node set. Nodes from this set would be first and last nodes of the 4-paths. Generally, the selection of the primary node set depends on the perspective of analysis. If the network is projected onto a one-mode network, the chosen node set is generally thought of as the primary one. In certain empirical cases, the selection of these nodes is obvious. For example, in scientific collaboration networks where researchers come together and co-author papers, the researchers are often assumed to be the primary nodes (Newman, 2001). However, in the case of interlocking directorates, it is not obvious whether directors or boards are the primary node set (Levine, 1979; Opsahl and Seierstad, 2009). This is due to the fact that tie formation is a mutual process where the directors must (1) be invited to join the board, and (2) accept the invitation.

The proposed coefficient represents only a first step in the process of redefining one-mode network measures for two-mode networks. Although the clustering coefficient is particularly affected by the projection procedure, there are many other measures that are also affected, such as the local clustering coefficient (Watts and Strogatz, 1998) and the array of structural holes measures (Burt, 1992). Redefining these measures for two-mode networks could increase their accuracy, and might give rise to novel insights.

Footnotes

1: In random networks where the present of each tie is given by a sought after density (i.e., where is the number of ties, and is the number of nodes), ties are independent of each other. However, ties are dependent upon each other in random networks with a set number of nodes and ties. For example, in an undirected random network, the likelihood of the first tie to be present is equal to the density. However, the likelihood of the second tie being present is dependent upon whether the first tie is present. Thus, it is equal to where is equal to 1 if the first was created and 0 otherwise. In fact, the presence of the last possible tie in a random network is given by the presence or lack thereof of the other ties in the network. Nevertheless, in large and sparse random networks, this dependence plays a negligible role as the number of actual ties is marginal relative to the number of possible ties.

2: I have knowingly chosen not to define the measure as the number of 4-cycles over the number of 3-paths. This is due to the fact that a 3-path would simply be, in the case of collaboration networks, the number of projects that a person’s collaborators are affiliated with. A 4-cycle would be, in that case, two individuals collaborating twice. Although this could be viewed as a form of clustering, it would not be triadic closure. In fact, it is a measure of reinforcement between two individuals rather than clustering of a group of individuals.

3: The 4-paths are 1-A-2-C-3 (closed); 1-A-2-C-5; 1-A-2-D-4; 1-B-3-C-2 (closed); 1-B-3-C-5; 2-A-1-B-3 (closed); 2-C-5-E-6; 3-C-2-D-4; 3-C-5-E-6; 4-D-2-C-5.

4: These 2-paths are: 1-2-3 (closed); 1-2-4; 1-2-5; 1-3-2 (closed); 1-3-5; 2-1-3 (closed); 2-5-6; 3-2-4; 3-5-6; 4-2-5.

5: The components of this express are the following. The density of the two-mode network, , is the likelihood that a tie is present. The square of is the likelihood that (1) a tie is present from the first node on a 4-path to a node, and (2) a tie is present from that node to the last node on the 4-path. The inner brackets, , is the likelihood that these two ties are not present. The power of the inner bracket, , ensures that , is the likelihood that no node connect the first and last node on the 4-path. As the two nodes that are on the 4-path cannot close it, 2 is subtracted from . By subtracting the inner bracket and its power from 1, the outcome is the likelihood that a 4-path is closed. This is the randomly expected value of Eq. 2.

6: Simulations conducted on ensembles of random networks with different number of nodes and ties produced outcomes that were not statistically different from the outcome attained with the above expression. Each ensemble contained 1,000 random networks.

7: To highlight the intrinsic level of clustering in this network, I randomised the two-mode structure while keeping the degree distributions (i.e., randomly assign the ties in the two-mode network while keeping each author’s number of co-authored papers, and each paper’s number of authors) before projecting it onto a one-mode network and calculating the clustering coefficient (Eq. 1). The average clustering coefficient across 100 random networks was 0.1235764, which is over 350 times larger than the coefficient found in a random one-mode network with the same number of nodes and ties.

8: Unless otherwise stated, the weighted clustering coefficients are based on the arithmetic mean method for defining the 2-path and 4-path values.

References

Burt, R.S., 1992. Structural Holes. Harvard University Press, Cambridge, MA.
Davis, A., Gardner, B. B., Gardner, M. R., 1941. Deep South. University of Chicago Press, Chicago, IL.
Erdo s, P., Rényi, A., 1959. On random graphs. Publicationes Mathematicae 6, 290-297.
Holland, P.W., Leinhardt, S., 1970. A method for detecting structure in sociometric data. American Journal of Sociology 76, 492-513.
Levine, J., 1979. Joint-space analysis of “pick-any” data: Analysis of choices from an unconstrained set of alternatives. Psychometrika 44(1), 85-92.
Luce, R.D., Perry, A.D., 1949. A method of matrix analysis of group structure. Psychometrika 14 (1), 95-116.
Mizruchi, M. S., 1996. What Do Interlocks Do? An Analysis, Critique, and Assessment of Research on Interlocking Directorates. Annual Review of Sociology 22, 271-298.
Newman, M. E. J., 2001. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E 64, 016132.
Opsahl, T., Colizza, V., Panzarasa, P., Ramasco, J.J., 2008. Prominence and control: The weighted rich-club effect. Physical Review Letters 101, 168702.
Opsahl, T., Panzarasa, P., 2009. Clustering in weighted networks. Social networks 31, 155-163.
Opsahl, T., Seierstad, C., 2009. For the few, not the many: The effects of affirmative action on presence, prominence, and social capital of female directors in Norway. Unpublished manuscript available on http://toreopsahl.com/publications/
Wasserman, S., Faust, K., 1994. Social Network Analysis: Methods and Applications. Cambridge University Press, New York, NY.
Watts, D.J., Strogatz, S.H., 1998. Collective dynamics of small-world networks. Nature 393, 440-442.
Zajac, E., 1988. Interlocking directorates as an interorganizational strategy: A test of critical assumptions. Academy of Management Journal 31(2), 428-438.

What to try it with your data

# Load tnet
library(tnet)

# This is the code for the binary two-mode network above
net <- cbind(
i=c(1,1,2,2,2,3,3,4,5,5,6),
p=c(1,2,1,3,4,2,3,4,3,5,5))

# Calculate measure
clustering_tm(net)

# This is the code for the weighted two-mode network above
net <- cbind(
i=c(1,1,2,2,2,3,3,4,5,5,6),
p=c(1,2,1,3,4,2,3,4,3,5,5),
w=c(3,5,6,1,2,6,2,1,3,1,2))

# Calculate measure
clustering_tm(net)

tnet is a package written in R that currently can calculate weighted social network measures, from version 0.1.0 analyse two-mode networks, and from version 0.2.0 detect underlying principles that guide tie formation in datasets with time-stamped ties. It forms part of a wider effort to analyse richer networks datasets without transforming the networks into simple static binary undirected one-mode ones. This post details tnet‘s capabilities related to weighted networks.

Motivation

Almost all of the ideas posted on this blog are related to weighted networks as, I believe, taking into consideration tie weights enable us to uncover and study interesting network properties. Not only are few network measures applicable to weighted networks, but there is also a lack of software programmes that can analyse this type of networks. To the best of my knowledge, there are no programmes that can both deal with weighted networks and allow users to create their own functions. On the one hand, programmes like UCINET and Pajek have a small set of functions for weighted networks, but they do not allow users to programme additional functions (Batagelj and Mrvar, 2007; Borgatti et al., 2002). Therefore, researchers proposing new measures must create stand-alone programmes to deal with a single aspect of weighted networks (e.g., Brandes, 2001; Newman, 2001; Opsahl et al., 2008; Opsahl and Panzarasa, 2009). On the other, a number of packages for analysing networks has been created within the open-source statistical programme R, notably the sna and statnet-packages (Butts, 2006; Handcock et al., 2003). These packages allow researchers to create additional functions on top of existing ones. This ability reduces the time spent on programming greatly, and let researchers focus on the contribution to the literature instead. For example, if someone has already written a function for identifying the shortest paths in a network, a researcher that would like to extend this measure can simply work on this code without programming the function from scratch. However, the sna and statnet-packages rely on the basic network-package for data structures to represent networks (Butts et al., 2008). This basic package does not have a data class for weighted networks. Therefore, to ease the development of new functions for weighted networks, a new platform is needed. tnet represent an attempt to create such a platform. Although it is a user-written package in R similar to the sna and statnet-packages, it does not rely on the network-package. It utilises its own data structures, one of which can handle weighted networks.

Data Structures

Since most networks are sparse (i.e., the number of ties (A) is much lower than the squared number of nodes (), I opted for an edgelist format instead of a matrix one. A binary edgelist consists of two columns that represent the pairs of nodes that are tied together in a network (e.g., the edgelist1-format in UCINET’s dl files; Borgatti et al., 2002). When a directed network is represented, the first column represents the nodes that create the ties, whereas the second column represents the target nodes. Thus, an edgelist is an table. This type of list has been extended to cover weighted networks by adding a third column representing the weight of the ties (an table; Borgatti et al., 2002). While the matrix format records the weight of all possible ties (a non-established tie would get a weight of 0), this format records the sender, receiver, and weight of established ties only. Thus, the space needed to store a network is proportional instead of . The main advantage of this format is that it can scale to networks with many nodes as it is the number of ties, not nodes, that determine the size of the data object. Although many programmes can read edgelists, most network analysis programmes rely on an internal matrix representation, e.g. UCINET and the network-package (Borgatti et al., 2002). Conversely, Pajek, which was designed to analyse large-scale sparse networks, specifically uses an internal edgelist representation (Batagelj and Mrvar, 2007). Thus, tnet can efficiently be applied to large-scale sparse networks.

In an effort to stay consistent with existing data structures, this three column table is also the structure used by tnet. The object class of an edgelist in R should be data.frame class. This class allows the different columns of a table to be of different classes, such as integer and numeric (i.e., real numbers, which takes more space than integers). Thus, the data.frame class is more efficient at storing data than the matrix class, which requires all columns to be numeric. The first two columns of the edgelist are assumed integers (i.e., the identification number of the node creating the tie and the identification number of the node receiving the tie, respectively). The third column can be integers or numeric that represents the weights attached to the ties.

To illustrate the edgelist structure, the figure below shows two networks with two ties each. The ties in the first network are directed, whereas in the second one they are undirected.

Example of a directed (a) and an undirected (b) network with weighted ties. This figure is based on Figure 16 in my thesis.

The directed network in the above figure should be represented by using the following table:

1   2   4
1   3   2

A network is deemed as undirected if all ties are included twice – one in each direction. The undirected network in the above figure should be represented by the following table:

1   2   4
2   1   4
1   3   2
3   1   2

There are a number of functions that help users to convert data in other formats into the weighted edgelist format. For example, if a dataset is undirected, but there is only one entry for each tie in the edgelist, the symmetrise-function adds a second entry of the edge with the identification numbers of the creator and target nodes reversed. Moreover, if a dataset is similar to an edgelist, but with only two columns (representing the identification numbers of the creator and target nodes) and multiple entries of the same tie refer to the weight of that tie (e.g., if a tie has a weight of 3, it is included three times), then the shrink_to_weighted_network-function allows the users to convert the edgelist into the correct format. To allow for a comparison between weighted and binary network measures, the dichotomise-function creates a binary network from a weighted one. It does so by removing the ties in a weighted edgelist that fall below a certain cut-off and sets the weight to 1 for the remaining ones.

Implemented Network Measures

The main measures that are implemented in tnet are generalised versions of the global clustering coefficient or transitivity (Opsahl and Panzarasa, 2009), local clustering coefficient (Barrat et al., 2004), and centrality measures (degree, closeness, and betweenness; Barrat et al., 2004; Brandes, 2001; Newman, 2001) as well as the weighted rich-club effect framework (Opsahl et al., 2008).

First, in Opsahl and Panzarasa (2009), we generalised the global clustering coefficient or transitivity, which measures the fraction of triplets that are part of triangles (or closed), to weighted networks. We generalised this measure by assigning a value to the triplets, and then taking the ratio between the total value of closed triplets and the total value of all triplets. This measure has been implemented in the function clustering_w.

Second, the local clustering coefficient has also been generalised to weighted networks (Barrat et al., 2004). This coefficient measures the density of nodes’ ego networks by taking the ratio between the number of ties that exist among a node’s contacts over the total possible number. The number of possible ties is equal to the number of triplets where the focal node is the middle node. Barrat et al. (2004) generalised the measure by assigning a value to each triplet, which was defined as the mean tie weight of the two ties that make up a triplet, and then took the ratio between the total value of closed triplets and the total value of all triplet values. In a similar spirit as the global clustering coefficient, I have proposed to use three additional methods for defining triplet values. This coefficient with all the four methods for defining triplet values is implemented in the function clustering_w_barrat.

Third, generalisations of Freeman’s (198) centrality measures, namely degree, closeness and betweenness, have been implemented as degree_w, closeness_w, and betweenness_w. The generalisations of degree and betweenness have previously been highlighted in posts on this blog, but closeness has yet to be mentioned. The original measure was formalised as the inverse distance to all other reachable nodes in the network. Since it relies on the calculation of shortest distances in the network, a first step towards extending it to weighted networks is to generalise how shortest distances are defined. I have covered one such method in the post average shortest distance in weighted networks, which was based on work by Dijkstra (1959) and Newman (2001). In fact, Newman (2001) has already extended the closeness measure using this method. He applied both the binary and weighted measures to a coauthorship network, and found that different authors had the highest scores. This highlights the importance of considering the weights, and suggests that a binary measure might not be a good proxy of a possible weighted one.

Fourth, I have included the weighted rich-club effect framework (Opsahl et al., 2008). However, due to the very general nature of the framework, I have only implemented two prominence parameters (degree and node strength), and two reshuffling procedures (weight reshuffling and weight & link reshuffling) in the function weighted_richclub. For code with additional reshuffling procedures and significance (error bars), see details in the post: Weighted Rich-club Effect: A more appropriate null model for scientific collaboration networks.

In addition, there are two functions that create random weighted networks. First, rg_w takes a set of properties, and according to these properties, it produces a random network. These properties included the number of nodes and ties, the range of weights, and whether the resulting network should be directed. Second, rg_reshuffling_w takes an observed network and randomises certain properties, such as the creator node, target node, or the location of the weights. In fact, the latter function is used by weighted_richclub when creating corresponding random networks.

Availability and Licensing

Compiled versions and the source code of tnet are available through the CRAN servers. If you are using the Windows version of R, you should be able to install tnet by going to the ‘Package’-menu in R, opening ‘Install package(s)’, selecting a server close to you, and then, choosing ‘tnet’ from the list. For specific details on how to install if you are unfamiliar with the CRAN system, see tnet‘s supporting website.

tnet, as the information on this blog, is published under the Creative Commons Attribution-Noncommercial 3.0-lisence. This means that you are free to:
· share (copy, distribute and transmit)
· remix (adapt)
under the following conditions:
· attribution (you must cite or link to it)
· noncommercial (you may not use it for commercial purposes).

The current citation for tnet is:

Opsahl, T., 2009. Structure and Evolution of Weighted Networks. University of London (Queen Mary College), London, UK, pp. 104-122. Available at http://toreopsahl.com/publications/thesis/.

References

Barrat, A., Barthelemy, M., Pastor-Satorras, R., Vespignani, A., 2004. The architecture of complex weighted networks. Proceedings of the National Academy of Sciences 101 (11), 3747-3752. arXiv:cond-mat/0311416

Batagelj, V., Mrvar, A., 2007. Pajek: Program for Large Network Analysis: version 1.20. http://pajek.imfm.si/.

Borgatti, S. P., Everett, M. G., Freeman, L. C., 2002. Ucinet for Windows: Software for Social Network Analysis. Analytic Technologies, Harvard, MA.

Brandes, U., 2001. A Faster Algorithm for Betweenness Centrality. Journal of Mathematical Sociology 25, 163-177.

Butts, C. T., 2006. sna-package: Package for Social Network Analysis. R package version 1.4.

Butts, C. T., Handcock, M. S., Hunter, D. R., 2008. network: Classes for Relational Data. http://statnet.org/.

Dijkstra, E. W., 1959. A note on two problems in connexion with graphs. Numerische Mathematik 1, 269-271.

Freeman, L. C., 1978. Centrality in social networks: Conceptual clarification. Social Networks 1, 215-239.

Handcock, M. S., Hunter, D. R., Butts, C. T., Goodreau, S. M., Morris, M., 2003. statnet: Software Tools for the Statistical Modeling of Network Data. http://statnetproject.org.

Newman, M. E. J., 2001. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E 64, 016132.

Opsahl, T., Colizza, V., Panzarasa, P., Ramasco, J. J., 2008. Prominence and control: The weighted rich-club effect. Physical Review Letters 101 (168702). arXiv:0804.0417.

Opsahl, T., Panzarasa, P., 2009. Clustering in weighted networks. Social Networks 31 (2), 155-163.

Weighted Rich-club paper

(out-)degree or node (out-)strength for (directed) networks

However, when prominence was defined as node strength, we could not use either of these methods for undirect networks (For directed networks, it is possible to redirect a node’s out-going ties randomly in the network, thus maintaining the node strength). Therefore, for undirected networks, we created directed networks by linking connected nodes with two directed ties. The weight of an undirected tie is duplicated to the two directed ones. A transformed network based on this process is is illustrated in the first panel of this figure. Then we reshuffled the weights attached to the outgoing ties of a node, thus, maintaining the sum of weights on outgoing ties. For example, in the second panel of the figure (the weight of a directed tie is placed close to the origin of the tie in this figure) we have randomly the weights of node E’s outgoing ties, 1 and 5. This method has been refered to as the directed weight reshuffling method (Serrano et al., 2007). However, this method breaks the weight symmetry of ties.

The appropriateness of this method for undirected networks depends on the research setting and how tie weights are defined (I would like to thank Tom Snijders for highlighting this during my viva). For example, its applicability to undirected transportation networks is justified by the typically directed nature of traffic flows (although the US airport network displays a high symmetry; Barrat et al., 2004). Conversely, in an undirected collaboration network this might not be appropriate. More generally, for one-mode projections of two-mode networks, it might be more appropriate to reshuffle the two-mode network before projecting it onto a one-mode network (see randomisation without structural zeros in Rao et al., 1996; Snijders, 1991).

To test this method, I have applied it to the scientific collaboration network used in the Weighted Rich-club paper. Although the two-mode structure of this network is not publically available, Mark Newman sent me a copy of it (Thanks!).

To calculate using this null model, I first randomised the two-mode network by reshuffling ties while maintaining authors’ and papers’ degree (similarly to the method proposed by Molloy and Reed, 1995, for one-mode networks). One step of this method is exemplified in the diagram below. I then projected the randomised two-mode network onto a weighted one-mode network using the same method in the paper (for more details, see my post on projecting two-mode networks). Finally, I calculated the weighted-rich club effect, , on the one-mode projections. The is the average over many random networks.

The result from this procedure is the displayed in the following diagram (the bars refer to the 95% confidence interval, see Section 3.2.2 of my thesis for more details):

Weighted rich-club coefficient for Newman's (2001) scientific collaboration network. is the average measured on 1,000 one-mode projections of randomised two-mode network.

The diagram suggest that, unlike what we found using the Directed weight reshuffle (see Section 3.3 of my thesis), when prominence is defined as having co-authored more than 20 articles, there is a negative and significant weighted rich-club effect. Thus, this finding is a substantiation of the speculation that in certain social networks a negative effect exists among the very prominent people due to some form of competition. This might account for the reluctance of highly productive authors to establish strong ties among themselves, as is suggested by the lack of interaction among the two most productive authors in the Figure below: Barabasi and Newman are not connected.¹

Subset of the prominent nodes () in the network science collaboration network (Newman, 2006). Only ties among the prominent nodes are shown. The size of the nodes corresponds to the strength or number of co-authored papers, and the width of each tie is proportional to its weight. This figure is adapted from Figure 8B in my thesis, which in turn was based on Figure 3B in Opsahl et al. (2008).

The null model used in this post could also be used in a number of other frameworks. For example, it could be used in the topological rich-club (Colizza, 2006; Zlatic et al., 2008). In particular, it might be more appropriate for Zlatic et al. (2008) as they defined prominence as node strength.
_____________________
¹ Although Albert-László Barabási and Mark Newman have written a book together with Duncan Watts, it did not form part of this dataset. Nevertheless, this collaboration would add only a weak tie between these authors.

Want to test it with your data?

The following code was used to produced the above diagrams:

# Load tnet
library(tnet)

# Download data
data(Newman.Condmat.95.99)

# Run function
weighted_richclub_tm(Newman.Condmat.95.99.net.2mode)

References

Barrat, A., Barthelemy, M., Pastor-Satorras, R., Vespignani, A., 2004. The architecture of complex weighted networks. Proceedings of the National Academy of Sciences 101 (11), 3747-3752. arXiv:cond-mat/0311416

Colizza, V., Flammini, A., Serrano, M. A., Vespignani, A., 2006. Detecting rich-club ordering in complex networks. Nature Physics 2, 110-115. arXiv:physics/0602134

Molloy, M., Reed, B., 1995. A critical point for random graphs with a given degree sequence. Random Structures and Algorithms 6, 161-180.

Newman, M. E. J., 2001. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E 64, 016132.

Newman, M. E. J., 2006. Finding community structure in networks using the eigenvectors of matrices. Physical Review E 76 (036104). arXiv:physics/0605087

Opsahl, T., Colizza, V., Panzarasa, P., Ramasco, J. J., 2008. Prominence and control: The weighted rich-club effect. Physical Review Letters 101 (168702). arXiv:0804.0417.

Rao, A. R., Jana, R., Bandyopadhyay, S., 1996. A markov chain monte carlo method for generating random (0, 1)-matrices with given marginals. Sankhya A 58, 225- 242.

Serrano, M. A., Boguna, M., Vespignani, A., 2007. Patterns of dominant flows in the world trade web. Journal of Economic Interaction and Coordination 2, 111-124. arXiv:0704.1225.

Snijders, T. A. B., 1991. Enumeration and simulation methods for 0-1 matrices with given marginals. Psychometrika 56 (3), 397-417.

Zlatic, V., Bianconi, G., Diaz-Guilera, A., Garlaschelli, D., Rao, F., Caldarelli, G., 2008. On the rich-club effect in dense and weighted networks. arXiv:0807.0793.

tnet’s support website

Publication > Thesis-page

Acknowledgements

The theme of this thesis is interdependence among elements. In fact, this thesis is not just a product of myself, but also of my interdependence with others. Without the support of a number of people, it would not have been possible to write. It is my pleasure to have the opportunity to express my gratitude to many of them here.

For my academic achievements, I would like to acknowledge the constant support from my supervisors. In particular, I thank Pietro Panzarasa for taking an active part of all the projects I have worked on. I have also had the pleasure to collaborate with people other than my supervisors. I worked with Vittoria Colizza and Jose J. Ramasco on the analysis and method presented in Chapter 2, Kathleen M. Carley on an empirical analysis of the online social network used throughout this thesis, and Martha J. Prevezer on a project related to knowledge transfer in emerging countries. In addition to these direct collaborations, I would also like to thank Filip Agneessens, Sinan Aral, Steve Borgatti, Ronald Burt, Mauro Faccioni Filho, Thomas Friemel, John Skvoretz, and Vanina Torlo for encouragement and helpful advice. In particular, I would like to thank Tom A. B. Snijders and Klaus Nielsen for insightful reading of this thesis and many productive remarks and suggestions. I have also received feedback on my work at a number of conferences and workshops. I would like to express my gratitude to the participants at these.

On a social note, I would like to thank John, Claudius, and my family for their continuing support. Without them I would have lost focus. My peers and the administrative staff have also been a great source of support. In particular, I would like to extend my acknowledgements to Mariusz Jarmuzek, Geraldine Marks, Roland Miller, Jenny Murphy, Cathrine Seierstad, Lorna Soar, Steven Telford, and Eshref Trushin.

Few network measures exist for two-mode networks, and therefore, these networks are often projected onto a one-mode network (only one type of nodes). This is done by selecting one of the sets of nodes and linking two nodes from that set if they were connected to the same node (of the other kind). This process is illustrated for the blue nodes of the above diagram in this diagram. For example, node E and node F are connected to the same red node, therefore, in the one-mode projection they are tied together.

Traditionally, the ties in projected one-mode networks do not have weights attached to them. However, recent empirical studies of two-mode networks has created a weighted one-mode network by defining the weights as the number of co-occurrences (e.g., the number of events two individuals have co-attended or the number of papers that two authors had collaborated on). To formally describe this method and ease the comparison among the different methods introduced in this post, this method can be formalised as: where where is the weight between node and node , and is the nodes of the other kind that node and node are connected to, their co-occurrences (e.g., the red nodes in the above diagram). In the sample network, the tie between node A and node B has a weight of 2 as these two nodes have connections to two common red nodes, whereas the tie between node A and node C has only a weight of 1 as these nodes have connections to merely one common red node.

Newman (2001) extended this procedure while working with scientific collaboration networks. He argued that the social bonds among scientist collaborating with few others on a paper were stronger than the bonds among scientists collaborating with many on a paper. He proposed to discount for the size of the collaboration by defining the weights among the nodes using the following formula: where is the number of authors on paper (e.g., the number of blue nodes connected to the red node ). In the context of scientific collaboration networks, this implies that if two scientists who only write a single paper together with no other co-authors get a weight of 1 (e.g., node B and node D). Moreover, if two scientists have written two papers together without any co-author, the weight of their tie would be 2 (e.g., node B and E). However, if the two scientists have a co-author, the weight on the tie between them is 0.5 (e.g., node A and node C). A more complicated example is the tie between node A and node B in the diagram. They have written two papers together: one without any other co-author and one with node C as a co-author. The first paper would give their tie a weight of 1, and the second tie would add 0.5 to the weight of this tie. Therefore, the weight attached to their tie is 1.5. By discounting for the number of blue nodes attached to the same red node, this methods creates a one-mode projection in which the strength of a node is equal to the number of ties originating from that node in the two-mode network (e.g. the sum of weights attached to ties originating from node A in the one-mode projection is 2, and node A is connected to two red nodes).

So far this post has only dealt with binary two-mode networks. However, weighted two-mode networks also exist. For example, in online forums, where the node sets are users and threads (or topics), and a tie between two nodes is established if a user posts a message to a thread, it is possible to differentiate the two-mode ties based on the number of messages or characters posted by users to a specific thread. Thus creating a weighted two-mode network. By including the weights in the two-mode network, the data used for analysis is richer than if they were discarded.

Note: The weights of the directed ties are placed close to the node that the ties originate from. For example, the tie from node B to node D has a weight of 4, whereas the tie from node D to node B has a weight of 6.

In a similar spirit as the method used by Newman (2001), it is also possible to discount for the number of nodes when projecting weighted two-mode networks. For example, it could be argued that if many online users post to a thread, their ties should be weaker than if there were few people posting to the thread. A straight forward generalisation is the following function: . This formula would create a directed one-mode network in which the out-strength of a node is equal to the sum of the weights attached to the ties in the two-mode network that originated from that node. For example, node C has a tie with a weight of 5 in the two-mode network and an out-strength of 5 in the one-mode projection.

Want to test it with your data?

Binary two-mode networks

The following code requires a binary two-mode network to be listed in an edgelist format with two columns named i and p. The i column refers to the nodes you would like to keep in the one-mode projection (e.g., the blue nodes), and the p column refers to the nodes you would like to discard (e.g., the red nodes). The binary two-mode network in the diagram above can be loaded using the following function.

net <- cbind(
i=c(1,1,2,2,2,2,2,3,4,5,5,5,6),
p=c(1,2,1,2,3,4,5,2,3,4,5,6,6))

The one-projections highlighted above can be created using the following code:

# Load tnet
library(tnet)

# Binary one-mode projection
projecting_tm(net, method="binary")

# Simply the number of common red nodes
projecting_tm(net, method="sum")

# Newman's method
projecting_tm(net, method="Newman")

Weighted two-mode networks

The following code requires a weighted two-mode network to be listed in an edgelist format with three columns named i, p, and w. The i column refers to the nodes you would like to keep in the one-mode projection (e.g., the blue nodes), the p column refers to the nodes you would like to discard (e.g., the red nodes), and the w column must be the weight of the ties. The weighted two-mode network in the diagram above can be loaded using the following function.

net.w <- cbind(
i=c(1,1,2,2,2,2,2,3,4,5,5,5,6),
p=c(1,2,1,2,3,4,5,2,3,4,5,6,6),
w=c(4,2,2,1,4,3,2,5,6,2,4,1,1))

The one-projections highlighted above can be created using the following code:

# Load tnet
library(tnet)

# Simply the sum of weights towards common red nodes
projecting_tm(net.w, method="sum")

# Generalisation of Newman's method
projecting_tm(net.w, method="Newman")

References

Davis, A., Gardner, B. B., Gardner, M. R., 1941. Deep South. University of Chicago Press, Chicago, IL.

Newman, M. E. J., 2001. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E 64, 016132.

Simmel’s argument was used as a key assumption by Granovetter (1973) when he formulated the Strength of Weak Ties theory. The theory states that weak ties in social networks (e.g., acquaintances as opposed to best friends) are more valuable than strong ties as they bring novel information to a person (ego). This assumption is based on the argument that ego’s close contacts are more likely to know each other (Simmel, 1923[1950]). This argument is illustrated in this sample network for node B. The node has two strong ties to two nodes that are connected, and two weak ties to nodes that are disconnected. Since the close contacts are connected, they are more likely to move in the same social circles as ego and each other than ego’s acquaintances. Thus, close friends’ knowledge is more likely to overlap with ego’s existing knowledge and each others’.

Following Granovetter, Burt (1992) and Coleman (1988) developed two sides of this theory. Burt developed the notion of structural holes, which states that a person benefits from being a broker among other disconnected contacts. On the contrary, Coleman argued that, although it might be beneficial to have many acquaintances when it comes to explicit knowledge, the same cannot be argued for tacit knowledge. A key factor in tacit knowledge transfer is close bonds, shared framework, and trust. These elements describe embedded relationships. Thus, ego must be embedded to optimise tacit knowledge transfer, and broker to optimise explicit knowledge transfer.

Simmel’s claim that strong social ties cannot exist without being embedded is key to this area of investigation; however, are triangles more likely to be constructed by strong ties than by weak ones?

Doreian (1969) and Opsahl and Panzarasa (2009) studied how the clustering coefficient in weighted networks changed when only considering ties with at least a certain value. As the cut-off value increased, the clustering coefficient generally decreased. However, it is not accurate to argue that strong ties is less embedded in triangles than weak ties based on such an analysis. This is due to the fact that the network structure changes when applying an increasinly restrictive cut-off as ties are removed. This implies that the networks on which the clustering coefficient is measured is not comparable.

A possible way of appropriately testing the above question is to compare the weighted clustering coefficient () to the clustering coefficient calculated on a binary network where all ties with a weight greater than 0 is set to present (). In so doing, the two coefficients are calculated on the same network structure. The only difference is that the former takes the weights of ties into consideration. This is possible due to the fact that if weights are randomly assigned in the network, the weighted clustering coefficient is equal to the binary coefficient.

If the ratio between the weighted clustering coefficient and the binary coefficient is higher than 1, it can be argued that triplets made up by strong ties are more likely to be closed than triplets made up by weak ties (). On the contrary, it can be argued that triplets made up by weak ties are more likely to be closed than those made up by strong ties if the ratio is less than 1.

Want to test it with your data?

First, you need to ensure that your data confirm to the tnet standard for weighted networks. Then you need to load it into an R session. The sample network above can be loaded using the following command:

# Load network as an object called net:
net <- cbind(
i=c(1,1,2,2,2,2,3,3,4,5,5,6),
j=c(2,3,1,3,4,5,1,2,2,2,6,5),
w=c(4,2,4,4,1,2,2,4,1,2,1,1))

Second, you need to download and install if you have not done so already for another function.

Third, by running the following commands, you will get the ratio between the weighted and binary clustering coefficient:

# Load tnet
library(tnet)

# Create a binary version of net
net.b <- dichotomise(net, GT=0)

# Calculate the clustering coefficient of the weighted network and of the binary network, and then, divided the former by the latter:
clustering_w(net, measure="am")/
clustering_w(net.b, measure="am")

Empirical tests

To empirically investigate whether strong ties are more likely to be part of closed triplets than weak ties, I have relied upon a set of publicly available datasets that I have collected.

Network		Nodes	Ties
1	Freeman’s EIES network (time 1)	48	695	0.7627	0.7702	1.0099
2	Freeman’s EIES network (time 2)	48	830	0.8131	0.8214	1.0102
3	Freeman’s EIES network (messages)	32	460	0.6386	0.7378	1.1555
4	Consulting (advice)	46	879	0.6932	0.7130	1.0285
5	Consulting (value)	46	858	0.6764	0.6852	1.0131
6	Research team (advice)	77	2228	0.6848	0.7127	1.0408
7	Research team (awareness)	77	2326	0.6785	0.6957	1.0253
8	C.elegans’ neural network	306	2345	0.1818	0.2364	1.3009
9	US airport network	500	2980	0.3514	0.4765	1.3562
10	Newman’s scientific collaboration network	16730	47594	0.3596	0.3178	0.8838

As it is possible to see from the above table, 9 of the 10 networks have a weighted clustering coefficient that is higher than the traditional clustering coefficient. This suggest that triangles generally are, in fact as suggested by Simmel (1923[1950]), made up by strong ties.

The tenth network, which shows the opposite result, is different from the other nine networks. It is a one-mode projection of a two-mode network, whereas the first nine networks are native one-mode networks. This feature could explain the observed result as the one-mode projection is constructed in a way that penalises the weight of ties within triangles, or more specifically, large fully connected cliques (Newman, 2001). See the description of the construction for more details. The next post will describe some methods for creating weighted one-mode projections of two-mode networks.

The code needed to created the above table was:

# Load tnet
library(tnet)

# Load networks
data(Freemans.EIES, Cross.Parker.Consulting, Cross.Parker.Manufacturing, celegans.n306, USairport.n500, Newman.Condmat.95.99)
net <- list()
net[[1]] <- Freemans.EIES.net.1.n48
net[[2]] <- Freemans.EIES.net.2.n48
net[[3]] <- Freemans.EIES.net.3.n32
net[[4]] <- Cross.Parker.Consulting.net.info
net[[5]] <- Cross.Parker.Consulting.net.value
net[[6]] <- Cross.Parker.Manufacturing.net.info
net[[7]] <- Cross.Parker.Manufacturing.net.aware
net[[8]] <- celegans.n306.net
net[[9]] <- USairport.n500.net
net[[10]] <- Newman.Condmat.95.99.net.1mode.wNewman

# Calculate values
output <- data.frame(CGT0=NaN, Cw=NaN, Ratio=NaN)
for(i in 1:length(net)) {
  output[i,"CGT0"] <- clustering_w(dichotomise(net[[i]]), measure="am")
  output[i,"Cw"] <- clustering_w(net[[i]], measure="am")
}
output[,"Ratio"] <- output[,"Cw"]/output[,"CGT0"]

References

Burt, R. S., 1992. Structural Holes: The Social Structure of Competition. Harvard University Press, Cambridge, MA.

Coleman, J. S., 1988. Social capital in the creation of human capital. American Journal of Sociology 94, S95-S120.

Doreian, P., 1969. A note on the detection of cliques in valued graphs. Sociometry 32 (2), 237-242.

Granovetter, M., 1973. The strength of weak ties. American Journal of Sociology 78, 1360-1380.

Opsahl, T., Panzarasa, P., 2009. Clustering in weighted networks. Social Networks 31 (2), 155-163.

Simmel, G., 1950. The Sociology of Georg Simmel (KH Wolff, trans.). Free Press, New York, NY.

Pietro Panzarasa

Social Networks

download the paper directly

preprint with the exact same text

Abstract

In recent years, researchers have investigated a growing number of weighted networks where ties are differentiated according to their strength or capacity. Yet, most network measures do not take weights into consideration, and thus do not fully capture the richness of the information contained in the data. In this paper, we focus on a measure originally defined for unweighted networks: the global clustering coefficient. We propose a generalization of this coefficient that retains the information encoded in the weights of ties. We then undertake a comparative assessment by applying the standard and generalized coefficients to a number of network datasets.

Motivation

In this sample network the binary clustering coefficient is 0.33 as a third of the triplets are closed by being part of a triangle. By looking at the weights, it is possible to see that the strongest ties are in part of the closed triplets. This is not reflected in the binary clustering coefficient.

By applying the proposed generalisation of the coefficient using the arithmetic mean method for defining triplet value, the clustering coefficient increases to 0.42. This increase of this coefficient from the binary coefficient is a reflection of the fact that the strongest ties are part of the closed triplets.

Want to test it with your data?

The clustering_w function in tnet allows you to test the generalised clustering coefficient on your own dataset.

For example, to test the clustering_w function on the sample network above, you can run the following code in R:

# Load tnet
library(tnet)

# Load network
net <- cbind(
i=c(1,1,2,2,2,2,3,3,4,5,5,6),
j=c(2,3,1,3,4,5,1,2,2,2,6,5),
w=c(4,2,4,4,1,2,2,4,1,2,1,1))

# Run function
clustering_w(net, measure=c("am", "gm", "ma", "mi"))

# The output is:
#       am        gm        ma        mi
#0.4166667 0.4361302 0.3750000 0.5000000

To test in on Freeman’s third EIES network from the datasets page, you can do the following:

# Load tnet
library(tnet)

# Load network
data(Freemans.EIES)

# Run function
clustering_w(Freemans.EIES.net.3.n32, measure=c("am", "gm", "ma", "mi"))

# The output is:
#0.7378310 0.7331536 0.7410959 0.7249982

In addition to number of biases (e.g., the informant inaccuracy bias; Bernard et al., 1984; see my thesis for a critic), survey instruments and direct observation methods are generally labour-intensive and difficult to administer. As a result, most networks collected using these methods are of a fairly limited size, often comprising only a few tens (e.g., Bernard et al., 1988) or hundreds (e.g., Fararo and Sunshine, 1964) of people.

This limitation has been overcome by using archival data sources instead of surveys. For example, the online social network of 1,899 people used in Patterns and Dynamics of Users’ Behaviour and Interaction: Network Analysis of an Online Community could only reasonably be collected since the social interactions were automatically recorded. Other social network papers using archival data sources include Kossinets and Watts (2006) and Uzzi and Spiro (2005).

Although archival data sources allow for larger networks to be collected, and in turn, more robust statistical analysis to be applied, a bias might be introduced into the data if information about the severing of ties is not included: archival data sources have a much better memory than individuals.¹ For a social network, this could imply that social interactions that are no longer relevant to an individual are recorded as being relevant. Moreover, the weight of ties might be overestimated. These issues do not exist when data is collected through surveys as each individual would only list current or relevant friends with the current tie strength (if they are honest that is).

In the empirical analysis of the online social network, we studied the network in two ways. First, we assumed that social ties never decay (the cumulative perspective). This assumes that if a social interaction is recorded on, for example, day 12, it will become included in the analysis from that point, and it will always remain included. Second, we followed Kossinets and Watts (2006) and imposed lifespans to the social relationships. This ensured that, if two people do not continue to communicate over time, their tie will be severed. This also applied to the weighted network: if the rate of messages sent from one person to another decreases, the tie would be weakened.

The length of the lifespan is crucial in determining which past events are taken into account to generate the network structure at a given point in time. By analysing which past events are relevant to the current state of the network, the length of the lifespan can be defined (Kossinets and Watts, 2006). An ill-defined lifespan will have the effect of, either breaking continuous social interactions into independent sets of interactions, or combining two separate interactions into a single one.

To illustrate the difference between imposing a lifespan and not imposing one, the following figure shows results from the the online social network where networks are constructed both cumulatively and with smoothing windows of 2, 3, and 6 weeks. Both panels in the figure highlight the vulnerability of network measures to the use of a smoothing window. Panel a suggests that there is only a small core of users that actively use the virtual community at the end of the observation period. An analysis of the cumulative network at that point would be heavily influenced by the majority of users that only used the network in the first 6 weeks, and would not reflect the current activities that are occurring in the community. This could bias network measures and, ultimately, the analysis. Panel b shows the evolution of one possible measure, the clustering coefficient. In particular, the clustering coefficient measured on the active core is mostly below the value found in the cumulative network.

The above figure also highlights the sensitivity of sampling time. By using shorter lifespans, the network measures become more unstable and dependent on the time at which the observation is taken. Kossinets and Watts (2006) argued that network measures would remain stable over time. As a result, the average of the measures in a given observation period can be generalised to a longer period of time. The figure, however, suggest that, when social relationships have a lifespan, network measures are not stable. Therefore, it is difficult to infer from network snapshots stable network measures that can reflect the network structure over a longer period of time.

By allowing for the severing of ties and sampling the network structure at various times over a longer period (e.g., each day in the observation period as we did for the online social network), the validity and robustness of a network analysis could be improved.
_____________________
¹ A number of other limitations, notably validity issues, could also be introduced into the data when using archival data sources.

Want to test it with your data?

First, you need to ensure that your data confirm to the tnet standard for longitudinal networks. Then you need to load it into an R session.

Second, you need to download, install, and load tnet.

Third, by combining the add_window_to_longitudinal_data-function and the longitudinal_data_to_edgelist-function, an instantaneous structure of the network at any point in time can be created.

# Add the severing of ties after 21 days
net <- add_window_to_longitudinal_data(net, window=21)

# Create the static network on February 20, 2009 at 7am
net0902200700 <- longitudinal_data_to_edgelist(net[net[,1]<="2009-02-20 07:00:00",])

Then you use the other functions to study the network.

# Average degree
tmp <- degree_w(net0902200700)
sum(tmp[,"degree"])/length(which(tmp[,"degree"]!=0))

# The global clustering coefficient
clustering_w(dichotomise(net0902200700))

References

Bernard, H. R., Killworth, P. D., Kronenfeld, D., Sailer, L. D., 1984. The problem of informant accuracy: the validity of retrospective data. Annual Review of Anthropology 13, 495-517.

Bernard, H. R., Kilworth, P. D., Evans, M. J., McCarty, C., Selley, G. A., 1988. Studying social relations cross-culturally. Ethnology 27 (2), 155-179.

Fararo, T. J., Sunshine, M., 1964. A Study of a Biased Friendship Network. Syracuse University Press, Syracuse, NY.

Kossinets, G., Watts, D. J., 2006. Empirical analysis of an evolving social network. Science 311, 88-90.

Uzzi, B., Spiro, J., 2005. Collaboration and creativity: The small world problem. American Journal of Sociology 111, 447-504.

Pietro Panzarasa

Kathleen M. Carley

Journal of the American Society for Information Science and Technology (JASIST)

download the paper directly

email

Abstract

This research draws on longitudinal network data from an online community to examine patterns of users’ behavior and social interaction and infer the processes underpinning dynamics of system use. The online community represents a prototypical example of a complex evolving social network in which connections between users are established over time by online messages. We study the evolution of a variety of properties since the inception of the system, including how users create, reciprocate, and deepen relationships with one another, variations in users’ gregariousness and popularity, reachability and typical distances among users, and the degree of local redundancy in the system. Results indicate that the system is a “small world” characterized by the emergence, in its early stages, of a hub-dominated structure with highly heterogeneous users’ behavior. We investigate whether hubs are responsible for holding the system together and facilitating information flow, examine first-mover advantages underpinning users’ ability to rise to system prominence, and uncover gender differences in users’ gregariousness, popularity, and local redundancy. We discuss the implications of the results for research on system use and evolving social networks, and for a host of applications, including information diffusion, communities of practice, and the security and robustness of information systems.

the Russian-Ukraine gas dispute

Formally Freeman (1978) defined the betweenness of a focal node to be:

where and are all the other nodes in the network, is the number of shortest paths between node and node , and is the number of those paths that go through the focal node.

In the sample network on the right, if the ties did not have a weight assigned to them, the grey lines represent the 9 shortest paths in the network that pass through intermediate nodes. The highlighted node is an intermediate on 8 of these paths. This will give this node a betweenness score of 8.

Brandes (2001) proposed a new algorithm for calculating betweenness faster. In addition to reducing the time, this algorithm also relaxed the assumption that ties had to be either present or absent (i.e. a binary network), and allowed betweenness to be calculated on weighted networks. This generalisation takes into account, that in weighted networks, the transaction between two nodes might be quicker along paths with more intermediate nodes that are strongly connected than paths with fewer weakly-connected intermediate nodes. This is due to the fact that the strongly connected intermediate nodes have, for example, more frequent contact than the weakly connected ones. For example, the tie between the top-left node and the focal node in the sample network above has four times the strength of the tie between the bottom-left node and the focal node. This could mean that top-left node has more frequent contact with the focal node than the bottom-left node has. In turn, this could imply that top-left node might give the focal node a piece of information (or a disease) four times quicker than the bottom-left node. If we are studying the nodes that are most likely to be funnelling information or diseases in a network, then the speed at which it travels, and routes that it takes, are clearly affected by the weights.

This post highlights the generalisation of Freeman’s (1978) betweenness measure to weighted networks by Brandes (2001). This generalisation is separate from the flow measure proposed by Freeman et al. (1991), which might be more appropriate in certain settings. The generalisation highlighted in this post follows the generalisation of closeness to weighted networks by Newman (2001) who built on work by Dijkstra (1959). Newman’s (2001) generalisation was introduced in detail in my previous post: a method for calculating the average shortest distance in weighted networks. Nevertheless, I will quickly reiterate Dijkstra’s (1959) and Newman’s (2001) work here:

1. Dijkstra (1959) proposed an algorithm to find the shortest paths in a network where the weights could be considered costs. The least costly path connecting two nodes was the shortest path between them (e.g. a network of roads where each leg of road has a time-cost assign to it).
2. Newman (2001) transformed the positive weights in a collaboration network into costs by inverting them (dividing 1 by the weight).
3. Based on the inverted weights, Newman (2001) applied Dijkstra’s algorithm and found the least-costly paths among all nodes.
4. The total cost of the paths from a node to all others was a measure of farness: the higher the number, the more it cost a node to reach all other nodes. To create a closeness measure, Newman (2001) followed Freeman (1978) and inverted the numbers (1 divided by the farness). Thus, a high farness was transformed into a low closeness, and a low farness was transformed into a high closeness.

The identification of the shortest paths in weighted networks outlined in the first three steps above can also be used when identifying the nodes which funnel transactions among other nodes in weighted networks. If we assume that transactions in a weighted network follow the shortest paths identified by Dijkstra’s algorithm instead of the one with the least number of intermediate nodes, then the number of shortest paths that pass through a node might change.

Want to test it with your data?

First, you need to download, install, and load tnet in R. Then, you need to load your data accordingly to the tnet standard. The standard is described in detail on tnet’s website along with examples on how to transfer data from other software packages, such as UCINET and Pajek. In addition, you can manually enter the weighted edgelist. For example, this sample network can be loaded using the following command:

net <- cbind(
i=c(1,1,2,2,2,2,3,3,4,5,5,6),
j=c(2,3,1,3,4,5,1,2,2,2,6,5),
w=c(4,2,4,1,4,2,2,1,4,2,1,1))

Freeman’s (1978) binary betweenness measure can be calculated by running the betweenness_w and dichotomise-functions:

betweenness_w(dichotomise(net))[[1]]
     vertex betweenness
[1,]      1           0
[2,]      2           8
[3,]      3           0
[4,]      4           0
[5,]      5           4
[6,]      6           0

To see the outcome of the proposed algorithm in this post, you simply need to run the betweenness_w-function alone:

betweenness_w(net)[[1]]
     vertex betweenness
[1,]      1           4
[2,]      2           8
[3,]      3           0
[4,]      4           0
[5,]      5           4
[6,]      6           0

Now, node 1 (A) has gotten betweenness score of 4 as well. This is because the indirect path from node B to node C through A is used instead of the direct connection.

Note: The betweenness_w-function relies upon the sp.between-function in the RBGL package. The sp.between-function currently does not find more than one shortest path connecting two nodes even if an equally short path exists. This limitation affects the betweenness_w-function.

UPDATE: There is now a boost C++ implementation of Brandes’ (2001) algorithm. I have not tested it, but as far as I can tell, it should be more efficient and without this limitation.

UPDATE: The boost C++ implementation of Brandes’ (2001) algorithm will be included in tnet from version 0.0.6. This algorithm finds multiple paths if they have exactly the same distance. For example, if one path is found over the direct tie with a weight of 1 (distance = 1/1 = 1) and a second path is through an intermediary node with two ties with weights of 2 (distance = 1/2 + 1/2 = 1), the two paths have exactly the same distance. However, if there is a third path through two intermediaries with three ties with weights of 3 (distance = 1/3 + 1/3 + 1/3), it does not exactly equal 1 as computers read these values as 0.3333333 and the sum of these values is 0.9999999. Therefore, this path is considered shorter than the other two paths (distance = 1).

References

Brandes, U., 2001. A Faster Algorithm for Betweenness Centrality. Journal of Mathematical Sociology 25, 163-177.

Dijkstra, E. W., 1959. A note on two problems in connexion with graphs. Numerische Mathematik 1, 269-271.

Freeman, L. C., 1978. Centrality in social networks: Conceptual clarification. Social Networks 1, 215-239.

Freeman, L. C., Borgatti, S. P., White, D. R., 1991. Centrality in valued graphs: A measure of betweenness based on network flow. Social Networks 13 (2), 141-154.

Newman, M. E. J., 2001. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E 64, 016132.

The way tie strength is recorded as weights (operationalisation of tie strength) can introduce subjective biases in the analysis of weighted networks that might invalidate the analysis. Therefore, a method that retains elements from the research question and setting should be applied as it affects the outcomes of weighted network measures.

A precise definition of tie strength often exists in non-social networks (e.g. the number of synapses and gap junctions), making the operationalisation of tie strength into weights straight forward. Conversely, in social networks, this tend not to be the case. Most social networks are collected through survey instruments (Wasserman and Faust, 1994). For example, a friendship network could be collected by asking people to designate others from a list or roaster as “know this person’s name”; “acquaintace”, or “friend”. By collecting social relations in such a way, two issues exist.

First, what is considered an “acquaintance” or a “friend” can differ considerably from one person to another. This bias is usually referred to as the informant inaccuracy bias (Bernard et al., 1984). A possible way of overcoming this bias is to design questions that are targeting the elements of a social tie (duration, emotional intensity, intimacy, and exchange of services; Granovetter, 1973). A question that does this is:

Please indicate how often you have turned to this person for information or advice on work-related topics in the past three months.

with the ordinal scale: 0, Do not know this person; 1, Never; 2, Seldom; 3, Sometimes; 4, Often; 5, Very Often. (This question was used by Cross and Parker, 2004, to collect an advice network.)

Second, by using an ordinal scale (associating 0, 1, 2, 3, 4, and 5 to the six levels of relationship), the differences among the levels might be misrepresented. In addition, with the ordinal scale used in the above question, answers are subjected to the inevitable bias that comes from the different ways in which different people assess duration. One possible way to overcome the scale issues is to design a scale that is closer to a ratio scale and reflects duration more accurately. For example, a better scale for the above question could be: 0, Do not know this person/Never; 1, Once; 3, Monthly; 6, Bi-weekly; 12, Weekly.

Researchers should carefully design questions and, in turn, the scale of weights, to overcome these issues. Marsden and Campbell (1984) conducted a comparative analysis of Granovetter’s (1973) four criteria for defining tie weights. They found that emotional intensity was a better indicator of strength of friendship than the other three criteria. Nevertheless, I believe that researchers should choose the appropriate measures of tie strength depending on the nature of the nodes and ties and, more generally, on the context of the research setting. In turn, such a scale is likely to yield a network dataset that is richer in information, more robust against potential inaccuracies emanating from subjective judgments, and more suitable to investigations that rely on weighted network measures.

References

Barrat, A., Barthelemy, M., Pastor-Satorras, R., Vespignani, A., 2004. The architecture of complex weighted networks. Proceedings of the National Academy of Sciences 101 (11), 3747-3752. arXiv:cond-mat/0311416

Bernard, H. R., Killworth, P. D., Kronenfeld, D., Sailer, L. D., 1984. The problem of informant accuracy: the validity of retrospective data. Annual Review of Anthropology 13, 495-517.

Cross, R., Parker, A., 2004. The Hidden Power of Social Networks. Harvard Business School Press, Boston, MA.

Granovetter, M., 1973. The strength of weak ties. American Journal of Sociology 78, 1360-1380.

Guimera, R., Mossa, S., Turtschi, A., Amaral, L. A. N., 2005. The worldwide air transportation network: Anomalous centrality, community structure, and cities’ global roles. Proceedings of the National Academy of Sciences 102, 7794-7799. arXiv:cond-mat/0312535

Luczkowich, J. J., Borgatti, S. P., Johnson, J. C., Everett, M. G., 2003. Defining and measuring trophic role similarity in food webs using regular equivalence. Journal of Theoretical Biology 220, 303321.

Marsden, P.V., Campbell, K.E., 1984. Measuring Tie Strength. Social Forces 63, 482-501.

Nordlund, C., 2007. Identifying regular blocks in valued networks: A heuristic applied to the St. Marks carbon flow data, and international trade in cereal products. Social Networks 29 (1), 59-69.

Pastor-Satorras, R., Vespignani, A., 2004. Evolution and Structure of the Internet. Cambridge University Press, New York, NY.

Wasserman, S., Faust, K., 1994. Social Network Analysis. Cambridge University Press, Cambridge, MA.

Watts, D. J., Strogatz, S. H., 1998. Collective dynamics of “small-world” networks. Nature 393, 440-442.

Two versions of this measure exist: the global and the local. The global version was designed to give an overall indication of the clustering in the network, whereas the local gives an indication of the embeddedness of single nodes. This post is concerned with the latter version, but I will briefly introduce the former quickly here.

The global clustering coefficient is based on triplets of nodes. A triplet is three nodes that are connected by either two (open triplet) or three (closed triplet) undirected ties. A triangle consists of three closed triplets, one centred on each of the nodes. The global clustering coefficient is the number of closed triplets (or 3 x triangles) over the total number of triplets (both open and closed). The first attempt to measure it was made by Luce and Perry (1949). This measure gives an indication of the clustering in the whole network (global), and can be applied to both undirected and directed networks (often called transitivity, see Wasserman and Faust, 1994, page 243). However, it cannot be applied to weighted networks.

For this sample network, the binary global clustering coefficient would be 3 over 9, or 0.33. The closed triplets are B→A←C; A→B←C; A→C←B; whereas the open triplets are A→B←D; A→B←E; C→B←D; C→B←E; D→B←E; B→E←F. As can be seen from the sample network, the strongest ties are inside the triangle. This is not captured by the binary coefficient as the weights are not considered. In the second chapter of my thesis, I proposed a generalisation of the global clustering coefficient to weighted networks (both undirected and directed). For this network, the outcome would be 0.44. The increase from the binary coefficient (0.33) reflects the tendency of strong ties to be part of closed triplets.

The local clustering coefficient is based on ego network density or local density (Scott, 2000; Uzzi and Spiro, 2005; Watts and Strogatz, 1998). For a node, this is the fraction of the number of present ties over the total number of possible ties between the node’s neighbours. Therefore, the outcome ranges between 0 and 1: 0 if no ties exist between the neighbours, and 1 if all possible ties exists. For undirected networks, the local clustering coefficient is formally defined as:

The number of possible ties between a node’s neighbours can be calculated by multiplying the number of neighbours by the number of neighbours minus 1.

This formula can be re-written in terms of the adjacency matrix whose entries is 1 if node i is connected with node j and 0 otherwise:

The main advantage of this version of the clustering coefficient is that a score is assigned to each node (local measure). This enables researchers to study associations between the coefficient and other nodal properties (e.g. Panzarasa et al., 2009) and perform regression analyses with the observations being the nodes of a network (e.g. Uzzi and Lancaster, 2004). However, this version of the clustering coefficient suffers from three major limitations. First, its outcome does not take into consideration the weight of the ties in the network. Second, the local clustering coefficient cannot be calculated on directed networks. Third, a negatively correlation with degree is often found in real-world networks. This is due to the fact that it is “easier” for a node with two neighbours to get a score of 1 (only one tie is need) than for a node with 10 neighbours (45 ties must be present).

For the sample network, nodes A and C would get a score of 1 since all possible ties among their neighbours are present, whereas node E would get a score of 0 since none of the possible ties among its neighbours are present. Nodes D and F would not get a value since the number of possible ties among their neighbours is 0 (if a node has less than 2 neighbours, the coefficient is undefined). For node B, one out of six possible ties is present, so the coefficient is 1/6 or 0.1667.

Barrat et al. (2004) proposed a generalisation of the local clustering coefficient to weighted networks by taking the weights explicitly into account. First, they assigned a triplet value to each triplet in the network based on the arithmetic mean (normal average). Then for each node, they summed the value of the closed triplets that were centred on the node and divided it by the total value of all triplets centred on the node. Formally, they proposed the following equation:

For the above sample network, all the nodes, except node B, would get the same outcome as they get the extreme values. Node B would get a value of 0.242. The difference between this value and the one obtain in the binary analysis (0.167) is a reflection of node B’s strong ties being directed towards neighbours that are themselves connected.

However, the arithmetic mean is insensitive to differences between the two tie weights as an extreme value can have a major impact on the triplet value. Therefore, I propose three additional methods for defining the triplet value. These were originally used in my proposed generalisation of the global clustering coefficient.

First, it can be defined as the geometric mean of the weights attached to the two ties. This method overcomes some of the sensitivity issues as a triplet made up by a tie with a low value and a tie with a high value will have a lower value than if the arithmetic mean were used.

Second, it can be defined as the maximum value of the two weights. This method offsets a low tie weight and makes a triplet with a strong tie and a weak tie equal to a triplet with two strong ties.

Third, it can be defined as the minimum value of the two weights. This method offsets a high tie weight by making triplets with a strong tie and a weak tie equal to triplets with two weak ties.

The table below highlights the differences between the methods of defining the triplet value (adopted from Chapter 2 of my thesis).

It is vital to use an appropriate method for defining the value of a triplet as this impacts on the outcome of the coefficient. The method should be chosen based on the research question as well as the way in which the strength of the ties are operationalised into weights. For example, in a network where the weights correspond to the level of flow, and a weak tie would act as a bottleneck, the minimum method might be most appropriate to use.

Want to test it with your data?

First, you need to create an edgelist of your network (see the tnet documentation). The edgelist for the sample network above can be entered into R using the following command:

net <- cbind(
i=c(1,1,2,2,2,2,3,3,4,5,5,6),
j=c(2,3,1,3,4,5,1,2,2,2,6,5),
w=c(4,2,4,4,1,2,2,4,1,2,1,1))

Then you can run the following code to calculate the local clustering coefficient using the four definitions for triplet value.

# Load tnet
library(tnet)

# Calculated all measures (including the binary one)
clustering_w_local(net, measure=c("am","gm","ma","mi","bi"))

The output should be something along these lines (am is the weighted local clustering coefficient (wlcc) using the arithmetic mean; gm is the wlcc using the geometric mean; ma is the wlcc using the maximum method; mi is the wlcc using the minimum method; and bi is the binary version of the wlcc):

     node        am        gm        ma        mi        bi
[1,]    1 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000
[2,]    2 0.2424242 0.2654092 0.1818182 0.3636364 0.1666667
[3,]    3 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000
[4,]    4       NaN       NaN       NaN       NaN       NaN
[5,]    5 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000
[6,]    6       NaN       NaN       NaN       NaN       NaN

Nodes 4 and 6 do not have a value as they have less than 2 neighbours. Nodes 1 and 3 get a value of 1 because all their neighbours are connected, whereas node 5 get a value of 0 because none of its neighbours are connected.

Node 2 is the interesting node as it does not get the extreme values. The local clustering coefficient obtained by using the four definitions of triplet differ. The choice of which definition to use should be based on the research question. Moreover, within a regression framework, the different values might be used as robustness checks.

Empirical test

I have also tested this extension on the network of the 500 busiest commercial airports in the US linked together by scheduled flights (Thanks Vittoria for uploading this network!).

The code for downloading and calculating the measures is:

# Load tnet
library(tnet)

# Load network
## Please cite Colizza et al. (2007) doi:10.1038/nphys560
data(USairport.n500)
net <- USairport.n500.net

# Since this network has are very large tie weights in this network, the geometric method in R fails. This limitation in R can be overcome by transforming the values.
# Note: This step is not required unless you receive warnings when running the function.
net[,3] <- (net[,3]/min(net[,3]))

# Calculate measure
output <- clustering_w_local(net, measure=c("am","gm","ma","mi","bi"))

# Find the average value for nodes with a degree greater than 1
colMeans(output[,c("am","gm","ma","mi","bi")], na.rm=T)
       am        gm        ma        mi        bi
0.7660205 0.7751321 0.7610040 0.7828600 0.7264579

References

Barrat, A., Barthelemy, M., Pastor-Satorras, R., Vespignani, A., 2004. The architecture of complex weighted networks. Proceedings of the National Academy of Sciences 101 (11), 3747-3752. arXiv:cond-mat/0311416

Colizza, V., Pastor-Satorras, R., Vespignani, A., 2007. Reaction-diffusion processes and metapopulation models in heterogeneous networks. Nature Physics 3, 276-282. arXiv:cond-mat/0703129

Feld, S. L., 1981. The focused organization of social ties. American Journal of Sociology 86, 1015-1035.

Friedkin, N. E., 1984. Structural cohesion and equivalence explanations of social homogeneiety. Sociological Methods and Research 12, 235-261.

Heider, F., 1946. Attitudes and cognitive organization. Journal of Psychology 21, 107-112.

Holland, P. W., Leinhardt, S., 1970. A method for detecting structure in sociometric data. American Journal of Sociology 76, 492-513.

Holland, P. W., Leinhardt, S., 1971. Transitivity in structural models of small groups. Comparative Group Studies 2, 107-124.

Louch, H., 2000. Personal network integration: Transitivity and homophily in strong-tie relations. Social Networks 22, 45-64.

Luce, R. D., Perry, A. D., 1949. A method of matrix analysis of group structure. Psychometrika 14 (1), 95-116.

Panzarasa, P., Opsahl, T., Carley, K. M., 2009. Patterns and dynamics of users’ behavior and interaction: Network analysis of an online community. Journal of the American Society for Information Science and Technology 60 (5), 911-932.

Scott, J., 2000. Social Network Analysis: A Handbook. Sage Publications, London, UK.

Snijders, T. A. B., 2001. The statistical evaluation of social network dynamics. Sociological Methodology 31, 361-395.

Snijders, T. A. B., Pattison, P. E., Robins, G. L., Handcock, M. S., 2006. New specifications for exponential random graph models. Sociological Methodology 35, 99-153.

Uzzi, B., Lancaster, R., 2004. Embeddedness and price formation in the corporate law market. American Sociological Review 69, 319-344.

Uzzi, B., Spiro, J., 2005. Collaboration and creativity: The small world problem. American Journal of Sociology 111, 447-504.

Wasserman, S., Faust, K., 1994. Social Network Analysis. Cambridge University Press, Cambridge, MA.

Watts, D. J., Strogatz, S. H., 1998. Collective dynamics of “small-world” networks. Nature 393, 440-442.

the introduction of my thesis

Second, the shortest distances have been used as an underlying metric in a number of measures, such as centrality ones (Freeman, 1978). One of these measures is closeness, which is inversely proportional to the distance from a node to all other nodes in the network. A number of researchers has found an association between the closeness to others and performance – but this post is not about applications, but about the global average shortest distances.

The shortest distance among nodes in a network is quite easy to calculate if you only have present or absent ties: you simply count the ties along the shortest path. If two nodes are directly connected: distance=1; and if they are not directly connected, but are connected through intermediaries, then it is the lowest number of intermediary nodes +1. For example, if the ties in this network did not have weights, the distance between node A and F would be 3. Below is a matrix showing the shortest distances among the nodes in the sample network.

   A  B  C  D  E  F
A NA  1  1  2  2  3
B  1 NA  1  1  1  2
C  1  1 NA  2  2  3
D  2  1  2 NA  2  3
E  2  1  2  2 NA  1
F  3  2  3  3  1 NA

The average shortest distance in this network is 1.8 (the mean of the matrix, diagonal excluded).

Difficulty occurs when ties are differentiated, as they are in a weighted network. For example, the tie between node A and B has twice the strength of the tie between node A and node C. This could mean that node A has more frequent contact with node B than with node C. In turn, this could imply that node A might give node B a piece of information (or a disease) twice as likely as node C. If we are looking at the diffusion of information or diseases in a network, then the speed that it travels, and routes that it takes, are clearly affected by the weights.

Let us look at the shortest path from node C to node B. The direct connection carries a weight of 1 only; however, the indirect connection through node A is composed of stronger ties. Therefore, a piece of information (or a disease) might be transmitted quicker through node A than directly.

Dijkstra (1959) proposed an algorithm that sum the cost of connections and find the path of least resistance. For example, GPS devices uses this algorithm by assigning a time-cost to each leg of the road, and then find the route that cost least in terms of time. This algorithm can also be used in social network analysis. Newman (2001) applied it to a collaboration network of scientists by inversing the tie weights (dividing 1 by the weight). This implies that a stronger tie gets a lower cost than a weaker tie. Below is the network from above with the inverted weights:

By applying Dijkstra’s algorithm, we find that the direct connection between node C and node B has a cost of 1, whereas the indirect connection via node A has a cost of 0.75 (1/2 + 1/4). Therefore, according to this algorithm, the information will travel faster through the indirect connection. Below is a matrix showing the cost of the least costly routes among the nodes in the sample network.

     A    B    C    D    E    F
A   NA 0.25 0.50 0.50 0.75 1.75
B 0.25   NA 0.75 0.25 0.50 1.50
C 0.50 0.75   NA 1.00 1.25 2.25
D 0.50 0.25 1.00   NA 0.75 1.75
E 0.75 0.50 1.25 0.75   NA 1.00
F 1.75 1.50 2.25 1.75 1.00   NA

The average of this matrix is 0.9833.

This application of Dijkstra’s algorithm is fine to use if we wish to rank the nodes in terms of closeness to others (node A is closer to the others than node C because it has a stronger tie to node B); however, what does an average distance of 0.98 mean?

To this end, I suggest to “normalise” the weights by the average weight in the network. The weights would then be:

By calculating Dijkstra’s algorithm using this normalisation, we get the following shortest distances among nodes:

       A      B      C      D      E      F
A     NA 0.5825 1.1650 1.1650 1.7475 4.0775
B 0.5825     NA 1.7475 0.5825 1.1650 3.4950
C 1.1650 1.7475     NA 2.3300 2.9125 5.2425
D 1.1650 0.5825 2.3300     NA 1.7475 4.0775
E 1.7475 1.1650 2.9125 1.7475     NA 2.3300
F 4.0775 3.4950 5.2425 4.0775 2.3300     NA

The average of this matrix is 2.29.

A unit of distance then refers to one step with the average weight in the network. The average of the matrix suggests that on average nodes are 2.29 steps with average tie weight away from each other. This measure would be comparable across networks with different ranges of tie weights.

If anyone has any suggestions for improvements, let me know!

Want to test it with your data?

First, you need to download tnet and install it in R.

You need to create an edgelist of your network (see the tnet documentation). The edgelist for the sample network here can be entered into R using the following command:

net  <- cbind(
i=c(1,1,2,2,2,2,3,3,4,5,5,6),
j=c(2,3,1,3,4,5,1,2,2,2,6,5),
w=c(4,2,4,1,4,2,2,1,4,2,1,1))

Then you can write the following commands:

# Load tnet
library(tnet)

# Create a binary version of the network
bnet <- dichotomise(net)

# Calculate the average distance in the binary network
mean(distance_w(bnet),na.rm=T)

# Calculate the average distance in the weighted network
mean(distance_w(net),na.rm=T)

References

Dijkstra, E. W., 1959. A note on two problems in connexion with graphs. Numerische Mathematik 1, 269-271.

Freeman, L. C., 1978. Centrality in social networks: Conceptual clarification. Social Networks 1, 215-239.

Milgram, S., 1967. The small world problem. Psychology Today 2, 60-67.

Newman, M. E. J., 2001. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E 64, 016132.

the Weighted Rich-club Effect

So to reiterate the paper quickly:
1) we define a set of prominent nodes
2) we measure the total sum of weights attached to ties among the prominent nodes (see panel a of the figure below)
3) we sum the strongest ties in the network, where is the number of ties among the prominent nodes (see panel b of the figure below)
4) we take the ratio between the sum from point 2 to the sum from point 3, . As a result, if the prominent nodes share the strongest ties in the network, . If we the prominent nodes do not share all of the strongest nodes, we get a value lower than 1.

Schematic representation of a weighted network, with size of nodes proportional to their prominence, and width of links to their weight indicated by the corresponding numbers. Several definitions of prominence can be considered. (a) The prominent nodes and the ties among them are highlighted, giving ties and . (b) The strongest ties of the network are highlighted (6), yielding the following value for the denominator of : . We thus obtain . Adapted from Opsahl et al. (2008).

However, a problem might exist. Some definitions of prominence could be associated with the strength of ties, such as the average weight of the ties originating from a node (if a set of nodes has on average stronger ties, of course it would have stronger ties among them – everything else being equal). So therefore, we divide obtained in the real network by obtained on simulated corresponding random networks, (see the paper for this part and Colizza et al., 2006).

Now, in an effort to create a local measure, we would have to retain the research question: do prominent nodes direct their strongest ties to one another?, but rewrite it in terms of a single node. So, I thought:
1) designate prominent nodes
2) for each node, sum the weights towards prominent nodes (if a node has three ties with weights of 1, 2 and 6, and the two ties with weight of 1 and 6 are directed towards prominent nodes, then the sum is 7).
3) In case the definition of prominence is associated with the weight of ties, we need to discount for the randomly expected value. To this end, we could divide the value obtained in point 2, to the randomly expected value (average weight of ties (which is 3 in this example) multiplied with the number of ties towards prominent nodes): 6.

Let me know what you guys think?

Want to test it with your data?

First, you need to load your network in R in tnet format, and ensure that it complies with the required standard (see the tnet documentation for more information on the network structure required). The documentation also contains information on exporting network data from other programmes. Then you need to define a promience vector consisting on 1′s and 0′s where 1 signify prominence and 0 non-prominence. This vector must be of the same length as the number of nodes in the network.

# Load tnet
library(tnet)

# Load the above sample network
net <- cbind(
i=c(1,1,2,2,2,2,3,3,4,5,5,6),
j=c(2,3,1,3,4,5,1,2,2,2,6,5),
w=c(4,2,4,4,1,2,2,4,1,2,1,1))

# Define prominence parameter (node 1 (A), 2 (B) and 3 (C) are designated as prominent)
prominence <- c(1,1,1,0,0,0)

# Run function
weighted_richclub_w_local(net, prominence)

The output table is like this:

     node    ratio
[1,]    1 1.000000
[2,]    2 1.454545
[3,]    3 1.000000
[4,]    4 1.000000
[5,]    5 1.333333
[6,]    6 1.000000

Nodes 1, 3, 4 and 6 get a value of 1 as they do not have a choice in their behaviour because they are not connected to both prominent and non-prominent nodes. However, node 2 and 5 do have a choice in how to distributed their efforts. For example, node 5 have two ties, one with a weight of 2 to a prominent node and one with a weight of 1 to a non-prominent node. Therefore, it can be said that node 5 preferentially direct attention to prominent nodes.

References

Colizza, V., Flammini, A., Serrano, M. A., Vespignani, A., 2006. Detecting rich-club ordering in complex networks. Nature Physics 2, 110-115. arXiv:physics/0602134

Opsahl, T., Colizza, V., Panzarasa, P., Ramasco, J. J., 2008. Prominence and control: The weighted rich-club effect. Physical Review Letters 101 (168702). arXiv:0804.0417.

Vittoria Colizza

Pietro Panzarasa

José Javier Ramasco

Physical Review Letters (PRL)

the paper is available

Abstract

Complex systems are often characterized by large-scale hierarchical organizations. Whether the prominent elements, at the top of the hierarchy, share and control resources or avoid one another lies at the heart of the global organization and functioning. Inspired by network perspectives, we propose a new general framework for studying the tendency of prominent elements to form clubs with exclusive control over the majority of a system’s resources. We explore associations between prominence and control in the fields of transportation, scientific collaboration, and online communication.

UPDATE: I have suggested an additional null model in Weighted Rich-club Effect: A more appropriate null model for scientific collaboration networks

Want to test it with your data?

The weighted_richclub function in tnet allows you to test the framework on your own dataset.

neurons

individuals

airports

trade

By recording the strength of ties, we can create a weighted network. However, these networks are more difficult to analyse than if ties were simply present or absent. I devoted a large part my Ph.D. thesis to the analysis of these networks.

Basic network definitions

Throughout this blog I make use of some network terminology beyond nodes and ties, and I would like to clarify my definition of these terms.

Undirected / directed network

Degree and Strength of nodes

The degree of a node is equal to the number of other nodes the node is connected to. For example, node E in the above undirected network has a degree of 2 since it is connected to nodes B and F. In a directed network, each node can have an out-degree and in-degree. Out-degree is the number of nodes that the node connect to, whereas in-degree is the number of nodes that connect to the node. In the directed network, node E would have an out-degree of 2, but only an in-degree of 1 since only node B has directed a tie towards it.

The strength of a node is equal to the sum of weights attached to the ties that connect a node to others. For node E in the undirected network, this would be equal to 3. In a similar fashion as degree, in a directed network, each node has an out-strength and an in-strength. Out-strength is the sum of weights attached to ties originating from the node, whereas in-strength is the sum of weights attached to ties directed toward the node. Therefore, node E would have an out-strength of 3, but only an in-strength of 2.

One-/two-mode networks

Two-mode network projected onto a one-mode weighted network

Although the two-mode structure contains a number of details of the network, few network measures exist for two-mode networks. Therefore, these networks are often projected onto a one-mode network (only one type of nodes). This is done by selecting one of the sets of nodes and linking two nodes from that set if they were connected to the same node (of the other kind). This process is illustrated for the blue nodes of the second part of the diagram above. For example, node E and node F are connected to the same red node, therefore, in the one-mode projection they are tied together.

References

Davis, A., Gardner, B. B., Gardner, M. R., 1941. Deep South. University of Chicago Press, Chicago, IL.

Lazega, E., 2001. The Collegial Phenomenon: The Social Mechanisms of Cooperation Among Peers in a Corporate Law Partnership. Oxford University Press, Oxford, UK.

Newman, M. E. J., 2001. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E 64, 016132.