Skip to content

Create a disk.frame containing only specified columns #337

Closed

Description

@lucazav

Is there a simply way to create a disk.frame containing only specified columns?
I suppose it could be done using the csv_to_disk.frame's inmapfn parameter in the following way:

inmapfn = function(chunk) {
cols_to_keep = c('col01', 'col02', ...)
chunk[, (names(chunk) %in% cols_to_keep)]
...
}

or something like that,
Maybe a specific parameter like colsToKeep into the cst_to_disk.frame function could be really useful.

Activity

  1. xiaodaigh commented on May 3, 2021

    @xiaodaigh
    Collaborator

    You can already do that since {disk.frame} uses {data.table}'s fread which provides the select= option

    library(disk.frame)
    #setup_disk.frame() # omitted since the data is tiny
    
    # write the iris dataset to csv
    data.table::fwrite(iris, "iris.csv")
    
    # read only the Species and Sepal.Length columns via `select = `
    iris.df = disk.frame::csv_to_disk.frame("iris.csv", select = c("Species", "Sepal.Length"))
    
    
    collect(iris.df)
    
    # Species Sepal.Length
    # 1:    setosa          5.1
    # 2:    setosa          4.9
    # 3:    setosa          4.7
    # 4:    setosa          4.6
    # 5:    setosa          5.0
    # ---                       
    #   146: virginica          6.7
    # 147: virginica          6.3
    # 148: virginica          6.5
    # 149: virginica          6.2
    # 150: virginica          5.9
    
  2. locked and limited conversation to collaborators on May 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions