lapply - Create new variables, based on current variables, conditionally based on info in 2nd data frame
我一直在做很多新项目,我正在做很多不熟悉的数据准备和管理。
我有两个数据框:1)非常大,有数千个观察和变量(df1),2)数据框列出了df1(df2)中变量子集的采集年份范围。我需要在df1中为df1中的大量变量/列创建一个新变量。为df1创建的新变量将检查是否存在值(1),收集的年份中不存在值(0),或者不存在值且年份超出列出的收集范围在df2('NA')。
我花了几天时间阅读了大量的
这是我可行的起始数据框:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | grp <- c('a', 'a', 'a', 'b', 'b') year <- c(1991, 1992, 1993, 2005, 2010) v1 <- c(20.5, 30.5, 29.6, 28.7, 26.1) v2 <- c(100.0, 101.5, 105.1, 'NA', 95.0) v3 <- c(47.2, 'NA', 'NA', 'NA', 'NA') df1 <- data.frame(grp = grp, year = year, v1 = v1, v2 = v2, v3 = v3) df1 grp year v1 v2 v3 a 1991 20.5 100 47.2 a 1992 30.5 101.5 NA a 1993 29.6 105.1 NA b 2005 28.7 NA NA b 2010 26.1 95 NA |
这是我的参考数据框,其中包含df1中的变量:
1 2 3 4 5 6 7 8 9 10 | vars <- c('v1', 'v2', 'v3') start <- c(1989, 2004, 1980) end <- c(2015, 2011, 1994) df2 <- data.frame(vars = vars, start = start, end = end) df2 vars start end v1 1989 2015 v2 2004 2011 v3 1980 1994 |
我一直在用'lapply()'学习简单的东西:
1 | test <- df1[paste0(vars, '.cov')] <- lapply(df1[vars], function(x) as.integer(x > 0)) |
我在R中写道,我认为是需要满足的条件类型。我将用书面英语叙述:
收集的一年中存在的值(1)
1 | if (!is.na(x)) { x <- 1 } |
在df2(0)中列出的范围内的一年中不存在值
1 | if (is.na(x) & year %in% seq(df2$start[df2$vars == names(df1[x]), ], df2$end[df2$vars == names(df1[x]), ], 1)) { x <- 0 } |
值不存在且年份超出df2('NA')中列出的收集范围
1 | if (is.na(x) & !(year %in% seq(df2$start[df2$vars == names(df1[x]), ], df2$end[df2$vars == names(df1[x]), ], 1))) { x <- 'NA' } |
我在语法和索引方面做得最好,但我们很快就走出了自己的舒适区。
运行条件检查后,所需的输出/修改后的df1应如下所示:
1 2 3 4 5 6 | grp year v1 v2 v3 v1.cov v2.cov v3.cov a 1991 20.5 100 47.2 1 1 1 a 1992 30.5 101.5 NA 1 1 0 a 1993 29.6 105.1 NA 1 1 0 b 2005 28.7 NA NA 1 0 NA b 2010 26.1 95 NA 1 1 NA |
我对各种解决方案持开放态度,但这似乎是可能的路径。再次感谢所有的帮助。我是一位经验丰富的R建模师/科学家,但在过去一个月里,我已经学到了很多数据准备,'data.table'和'dplyr',并提供了所有帮助。
使用data.table:
1 2 3 4 5 6 7 8 9 10 | library(data.table) setDT(df1) DT = melt(df1, id = c("grp","year"), meas = patterns("^v"))[, value := type.convert(as.character(value))] # mark based on whether found or not within collection periods DT[df2, on=.(variable = vars, year >= start, year <= end), found := as.integer(!is.na(value))] # also mark if found outside collection periods DT[!is.na(value) & is.na(found), found := 1L ] |
这使
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | grp year variable value found 1: a 1991 v1 20.5 1 2: a 1992 v1 30.5 1 3: a 1993 v1 29.6 1 4: b 2005 v1 28.7 1 5: b 2010 v1 26.1 1 6: a 1991 v2 100.0 1 7: a 1992 v2 101.5 1 8: a 1993 v2 105.1 1 9: b 2005 v2 NA 0 10: b 2010 v2 95.0 1 11: a 1991 v3 47.2 1 12: a 1992 v3 NA 0 13: a 1993 v3 NA 0 14: b 2005 v3 NA NA 15: b 2010 v3 NA NA |
(
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | setDT(df1) setDT(df2) for (v in unique(df2$vars)){ df1[, (v) := type.convert(as.character(get(v)))] fcol = paste0("found.",v) df1[df2[vars == v], on=.(year >= start, year <= end), (fcol) := as.integer(!is.na(get(v)))] df1[!is.na(get(v)) & is.na(get(fcol)), (fcol) := 1L ] } grp year v1 v2 v3 found.v1 found.v2 found.v3 1: a 1991 20.5 100.0 47.2 1 1 1 2: a 1992 30.5 101.5 NA 1 1 0 3: a 1993 29.6 105.1 NA 1 1 0 4: b 2005 28.7 NA NA 1 0 NA 5: b 2010 26.1 95.0 NA 1 1 NA |