Reoccuring problem with gof-function

[Laupheim]

Hello,

unfortunately the gof-function is not working again. I used the updated version of btergm today. Here is what I got when I runned the script altogether with the data that I have send some days ago for error replication:

Starting GOF assessment on a single computing core....

No 'target' network(s) provided. Using networks on the left-hand side of the model formula as observed networks.

Simulating 50 networks from the following formula:
networks[[1]] ~ edges + mutual + twopath + ctriple + nodeocov("Psychological Safety") + nodeicov("Psychological Safety") + nodeocov("Psychological Empowerment") + nodeicov("Psychological Empowerment") + offset(edgecov(offsmat[[1]]))

Simulating 50 networks from the following formula:
networks[[2]] ~ edges + mutual + twopath + ctriple + nodeocov("Psychological Safety") + nodeicov("Psychological Safety") + nodeocov("Psychological Empowerment") + nodeicov("Psychological Empowerment") + offset(edgecov(offsmat[[2]]))

Simulating 50 networks from the following formula:
networks[[3]] ~ edges + mutual + twopath + ctriple + nodeocov("Psychological Safety") + nodeicov("Psychological Safety") + nodeocov("Psychological Empowerment") + nodeicov("Psychological Empowerment") + offset(edgecov(offsmat[[3]]))

3 networks from which simulations are drawn were provided.

Processing statistic: Triad census
Skipping statistic for the following reason: Error in 0:(nrow(mat) - 2): Argument der Länge 0
Processing statistic: Modularity (fast & greedy)
Processing statistic: Tie prediction

summary(g0)
Length Class Mode
Triad census 4 boxplot list
Modularity (fast & greedy) 4 univariate list
Tie prediction 9 rocpr list

Since I have got a proper output 3 days ago with the same data and code script, I do not think that the problem is caused by either of it.

[Laupheim]

Okay, so I have tried a few things now, and it seems that de-installing R and re-installing it, altogether with installing btergm from install_github("leifeld/btergm") does the trick. However, after I close and restart R, the error occurs again when I am running the code and loading btergm from the library.

So the problem might be caused by reloading the package with

library("btergm")

after I closed and restarted R. But I have no idea why this is.

Furthermore, the gof-function is also not working with nodefactor.

I hope this information is helpful.

[leifeld]

It sounds to me like you keep installing an old version of btergm for some reason. Perhaps you can diagnose the problem by printing the results of packageVersion("btergm") at the relevant stages when it does not work. It should show version 1.9.9.

Alternatively, if you keep running into problems with your local installation, you may want to use RStudio Cloud, where you can install the relevant packages in an online instance of RStudio. I sometimes recommend this to users whose local installation seems to cause problems.

[leifeld]

Closing because no further response. Please feel free to open a new issue if there are still any problems.

[NGneery]

Hello, I experienced a similar problem as Laupheim while using the gof function in btergm. Whatever gof statistics I requested, the gof() function only ever outputs geodesic distances, the FPR/TPR and Modularity (walktrap) statistics.

I already de- and reinstalled R (newest release 4.3.2) and reinstalled btergm (packageVersion("btergm") =‘1.10.11’) via install_github("leifeld/btergm"). I would be super grateful for any tips on how to solve this, thank you!

Here are 3 requests as examples:

1. request:
   gof_m3 <- gof(m3)

---

Starting GOF assessment on a single computing core....

Initial dimensions of the network and covariates:
t=1 t=2 t=3 t=4 t=5 t=6
iat_snet (row) 185 185 185 185 185 185
iat_snet (col) 185 185 185 185 185 185
conflict_snet (row) 185 185 185 185 185 185
conflict_snet (col) 185 185 185 185 185 185
alliance_snet (row) 185 185 185 185 185 185
alliance_snet (col) 185 185 185 185 185 185

Same dimensions but different labels across networks within time steps.
Trying to auto-adjust the dimensions of the networks. If this fails, provide conformable matrices or network objects.

All nodes are retained.

Number of nodes per time step after adjustment:
t=1 t=2 t=3 t=4 t=5 t=6
maximum deleted nodes (row) 0 0 0 0 0 0
maximum deleted nodes (col) 0 0 0 0 0 0
remaining rows 185 185 185 185 185 185
remaining columns 185 185 185 185 185 185

Dimensions of the network and covariates after adjustment:
t=1 t=2 t=3 t=4 t=5 t=6
iat_snet (row) 185 185 185 185 185 185
iat_snet (col) 185 185 185 185 185 185
conflict_snet (row) 185 185 185 185 185 185
conflict_snet (col) 185 185 185 185 185 185
alliance_snet (row) 185 185 185 185 185 185
alliance_snet (col) 185 185 185 185 185 185

No 'target' network(s) provided. Using networks on the left-hand side of the model formula as observed networks.

Simulating 100 networks from the following formula:
iat_snet[[1]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T) + edgecov(conflict_snet[[1]]) + edgecov(alliance_snet[[1]]) + nodematch("mil_burden_cat", diff = T) + nodematch("gdp_cat", diff = T) + absdiff("democracy") + nodeofactor("mil_burden_cat", levels = -1) + nodeifactor("mil_burden_cat", levels = -1) + nodeofactor("gdp_cat", levels = -1) + nodeifactor("gdp_cat", levels = -1) + nodeocov("ln_population") + nodeicov("ln_population") + nodeocov("democracy") + nodeicov("democracy") + nodeocov("sov_score") + nodeicov("sov_score") + nodeofactor("territory_95") + nodeifactor("territory_95") + nodeofactor("coup") + nodeifactor("coup") + nodeofactor("d_enemies") + nodeifactor("d_enemies")

...

6 networks from which simulations are drawn were provided.

Skipping statistic for the following reason: Error in 0:(nrow(mat) - 2): Argument der Länge 0
Skipping statistic for the following reason: Error in 0:(nrow(mat) - 2): Argument der Länge 0
Skipping statistic for the following reason: Error in as.edgelist.sna(dat): as.edgelist.sna input must be an adjacency matrix/array, edgelist matrix, network, or sparse matrix, or list thereof.

Skipping statistic for the following reason: Error in idegree(0:(nrow(mat) - 1)): konnte Funktion "idegree" nicht finden
Processing statistic: Geodesic distances
Processing statistic: Tie prediction
Processing statistic: Modularity (walktrap)

---

I then tried a less computationally demanding one just to see how it'd turn out.
2. request:
gof_m1 <- gof(m1, nsim = 25, statistics = c(esp, geodesic, ideg, odeg), parallel = "snow", ncpus = 2)

---

Starting GOF assessment using parallel processing on 2 cores....

No covariates provided.

All networks are conformable.

No 'target' network(s) provided. Using networks on the left-hand side of the model formula as observed networks.

Simulating 10 networks from the following formula:
iat_snet[[1]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 10 networks from the following formula:
iat_snet[[2]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 10 networks from the following formula:
iat_snet[[3]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 10 networks from the following formula:
iat_snet[[4]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 10 networks from the following formula:
iat_snet[[5]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 10 networks from the following formula:
iat_snet[[6]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

6 networks from which simulations are drawn were provided.

Skipping statistic for the following reason: Error in 0:(nrow(mat) - 2): Argument der Länge 0
Skipping statistic for the following reason: Error in 0:(nrow(mat) - 2): Argument der Länge 0
Skipping statistic for the following reason: Error in as.edgelist.sna(dat): as.edgelist.sna input must be an adjacency matrix/array, edgelist matrix, network, or sparse matrix, or list thereof.

Skipping statistic for the following reason: Error in idegree(0:(nrow(mat) - 1)): konnte Funktion "idegree" nicht finden
Processing statistic: Geodesic distances
Processing statistic: Tie prediction
Processing statistic: Modularity (walktrap)

---

3. request:
   gof_m2 <- gof(m2, nsim = 25, statistics = c(esp, geodesic, ideg, odeg), parallel = "snow", ncpus = 2)

---

Starting GOF assessment using parallel processing on 2 cores....

No covariates provided.

All networks are conformable.

No 'target' network(s) provided. Using networks on the left-hand side of the model formula as observed networks.

Simulating 25 networks from the following formula:
iat_snet[[1]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 25 networks from the following formula:
iat_snet[[2]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 25 networks from the following formula:
iat_snet[[3]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 25 networks from the following formula:
iat_snet[[4]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 25 networks from the following formula:
iat_snet[[5]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

Simulating 25 networks from the following formula:
iat_snet[[6]] ~ edges + gwesp(0.5, fixed = T) + gwodegree(0.5, fixed = T) + gwidegree(0.5, fixed = T)

6 networks from which simulations are drawn were provided.

Skipping statistic for the following reason: Error in 0:(nrow(mat) - 2): Argument der Länge 0
Processing statistic: Geodesic distances
Skipping statistic for the following reason: Error in idegree(0:(nrow(mat) - 1)): konnte Funktion "idegree" nicht finden
Skipping statistic for the following reason: Error in odegree(0:(nrow(mat) - 1)): konnte Funktion "odegree" nicht finden

[leifeld]

Can you provide a minimal, self-contained, reproducible example? Then I can try to diagnose the problem. Thanks.

[NGneery]

Sorry for having forgotten to report back on this issue. After a day, the gof() function worked for me again so it skipped my mind. Now I want to document my observations and troubleshooting approaches in case anyone else runs into the same problem.

For me, the bug kept reoccurrin in the past year. Sometimes btergm::gof() works just fine, sometimes it kept refusing to output specific stats saying it could not find the function for the stat or that some argument had length 0. Mostly, the stats concerned were esp, dsp or degree related. Other stats were outputted reliably, such as triads or prroc.

My initial troubleshooting approaches were based on Laupheim's and Philip's suggestions:

1. check btergm package version to see whether it was the most up to date one - however, each time the newest btergm version was already installed on my computer, so this is likely not the issue.
2. notwithstanding, proceed to de- and reinstall btergm - sometimes directly from leifeld's github repo.
3. if desperate, de- and reinstall R altogether.

Sometimes the problem vanished without doing anything, sometimes it was solved after trying one or all of the approaches outlined, and sometimes nothing seemed to help. Sometimes, just waiting a few days and the problem disappeared by itself, or sometimes simply using another computer works.
Anyhow, I could never pinpoint what the ultimate solution was, despite having systematically tried out all of them.

A few days ago, I noticed the following:
Close and reopen R. Load SOLELY "btergm" from the library - nothing else!
My guess is the gof() function might be masked or disturbed by one of the other packages but I do not know which one was responsible.
There could also simply be a problem with my computer.

If anyone else stumbles upon this issue, I hope the approaches suggested help. Usually, just waiting for a few days helps - though I was never able to find out why.

[leifeld]

A few thoughts on possible reasons and things to try:

Have you tried loading the statnet package followed by btergm at the beginning of the script? It could be that loading one of the statnet sub-packages, like network or sna, after btergm masks these functions.

Have you been using another package with the same function names? Maybe igraph masks some functions, not sure.

In the past, I noticed some weirdness with the version of the ergm package. There was some bug in one of the previous ergm versions, and depending on which version precisely was installed, btergm gof worked or failed. It needs to have the most recent version.

If you find out precisely under which conditions it happens or if you can even provide a reliable example, I would be grateful because that would allow me to investigate and fix the problem. Thanks.

[NGneery]

Thank you for your answer. I investigated some more, following your suggestions. A few findings:

1. order of loading packages: After having loaded btergm, I proceeded to load a selection of other packages, one by one. Each time after loading a new package, I tried using btergm::gof() to check which might mask the btergm::gof() function without informing the user. Then I detach all packages apart from btergm, and then restarted the procedure. In the end, none of the suspected were responsible (i.e. I tried igraph, ergm, sna, network, tergm, tsna, statnet.common, networkDynamic, intergraph). The order in which they were loaded (before or after btergm) did not cause any problem to btergm::gof() either.

2. other packages with the same function name: I normally use the namespace operator, so this is usually not a problem.

3. finally, I applied the same procedure as in 1), this time on my staple tools. I found out that Hmisc is the triggering factor. It does not matter whether Hmisc is loaded before or after btergm, the problem occurs reliably. Detaching Hmisc afterwards does not solve the problem, once this package is loaded into the global environment, gof() functioning is permanently impaired. btergm::gof() can no longer calculate statistics related to shared partners or degrees. Then, only closing and reopening R helps. If Hmisc is never retrieved during the new session, gof() works just fine.

4. Warnings: The user gets none if Hmisc is loaded before btergm. Loading Hmisc after btergm will produce this information:

Registered S3 method overwritten by 'htmlwidgets':
method from
print.htmlwidget tools:rstudio
Registered S3 method overwritten by 'data.table':
method from
print.data.table
Registered S3 method overwritten by 'Hmisc':
method from
summary.formula ergm

Attache Paket: ‘Hmisc’
Die folgenden Objekte sind maskiert von ‘package:base’:
format.pval, units

Also, this message sometimes appears when trying to use btergm::gof() after having loaded Hmisc (but not always):

Warnung: ungenutzte Verbindung 4 (<-view-localhost:11391) geschlossen
Warnung: ungenutzte Verbindung 3 (<-view-localhost:11391) geschlossen

OR this:

Warnung in for (i in seq_along(cenv$extra)) {
ungenutzte Verbindung 4 (<-view-localhost:11527) geschlossen
Warnung in for (i in seq_along(cenv$extra)) {
ungenutzte Verbindung 3 (<-view-localhost:11527) geschlossen

5. Minimal reproducible example

library(statnet) 
library(xergm.common)
library(btergm)
set.seed(9)

# example from Leifeld et al. (2018)
data("alliances", package = "xergm.common")
model <- btergm(allyNet ~ edges + gwesp(0, fixed = TRUE) + edgecov(LSP) + edgecov(warNet) + nodecov("polity") + nodecov("cinc") + absdiff("polity") + absdiff("cinc") + edgecov(contigMat), R = 5, parallel = "snow", ncpus = 2)

# before loading Hmisc =)
gof_1 <- btergm::gof(model, nsim = 5, parallel = "snow", ncpus = 2,
              statistics = c(ideg, odeg, esp, dsp, istar, ostar))

...
20 networks from which simulations are drawn were provided.

Processing statistic: Indegree
Processing statistic: Outdegree
Processing statistic: Edge-wise shared partners
Processing statistic: Dyad-wise shared partners
Processing statistic: Incoming k-star
Processing statistic: Outgoing k-star

plot(gof_1) # produces visualization with all stats requested

# after loading Hmisc =(
library(Hmisc)
gof_2 <- btergm::gof(model, nsim = 5, parallel = "snow", ncpus = 2,
              statistics = c(ideg, odeg, esp, dsp, istar, ostar))
plot(gof_2) # produces error message

Fehler in plot.gof(gof_2) : 'x' does not contain any gof objects.

Hope this helps.