Creates a corpus object from available sources. The currently available sources are:
a character vector, consisting of one document per element; if the elements are named, these names will be used as document names.
a data.frame, whose default document id is a variable identified
by docid_field; the text of the document is a variable identified by
textid_field; and other variables are imported as document-level
meta-data. This matches the format of data.frames constructed by the the
readtext package.
a tm VCorpus or SimpleCorpus class object, with the fixed metadata fields imported as docvars and corpus-level metadata imported as metacorpus information.
a corpus object.
corpus(x, ...) # S3 method for corpus corpus(x, docnames = quanteda::docnames(x), docvars = quanteda::docvars(x), metacorpus = quanteda::metacorpus(x), compress = FALSE, ...) # S3 method for character corpus(x, docnames = NULL, docvars = NULL, metacorpus = NULL, compress = FALSE, ...) # S3 method for data.frame corpus(x, docid_field = "doc_id", text_field = "text", metacorpus = NULL, compress = FALSE, ...) # S3 method for kwic corpus(x, ...) # S3 method for Corpus corpus(x, metacorpus = NULL, compress = FALSE, ...)
| x | a valid corpus source object |
|---|---|
| ... | not used directly |
| docnames | Names to be assigned to the texts. Defaults to the names of
the character vector (if any); |
| docvars | a data.frame of document-level variables associated with each text |
| metacorpus | a named list containing additional (character) information
to be added to the corpus as corpus-level metadata. Special fields
recognized in the
|
| compress | logical; if |
| docid_field | column index of a document
identifier; defaults to |
| text_field | the character name or numeric index of the source
|
A corpus-class class object containing the original texts, document-level variables, document-level metadata, corpus-level metadata, and default settings for subsequent processing of the corpus.
The texts and document variables of corpus objects can also be
accessed using index notation. Indexing a corpus object as a vector will
return its text, equivalent to texts(x). Note that this is not the
same as subsetting the entire corpus -- this should be done using the
subset method for a corpus.
Indexing a corpus using two indexes (integers or column names) will return
the document variables, equivalent to docvars(x). It is also
possible to access, create, or replace docvars using list notation, e.g.
myCorpus[["newSerialDocvar"]] <-
paste0("tag", 1:ndoc(myCorpus)).
For details, see corpus-class.
A corpus currently consists of an S3 specially classed list of elements, but you should not access these elements directly. Use the extractor and replacement functions instead, or else your code is not only going to be uglier, but also likely to break should the internal structure of a corpus object change (as it inevitably will as we continue to develop the package, including moving corpus objects to the S4 class system).
corpus-class, docvars, metadoc,
metacorpus,
settings, texts, ndoc,
docnames
# create a corpus from texts corpus(data_char_ukimmig2010)#> Corpus consisting of 9 documents and 0 docvars.# create a corpus from texts and assign meta-data and document variables summary(corpus(data_char_ukimmig2010, docvars = data.frame(party = names(data_char_ukimmig2010))), 5)#> Corpus consisting of 9 documents, showing 5 documents. #> #> Text Types Tokens Sentences party #> BNP 1125 3280 88 BNP #> Coalition 142 260 4 Coalition #> Conservative 251 499 15 Conservative #> Greens 322 679 21 Greens #> Labour 298 683 29 Labour #> #> Source: /Users/kbenoit/Dropbox (Personal)/GitHub/quanteda/docs/reference/* on x86_64 by kbenoit #> Created: Thu Aug 10 12:42:43 2017 #> Notes: #>#> Corpus consisting of 14 documents and 0 docvars.# import a tm VCorpus if (requireNamespace("tm", quietly = TRUE)) { data(crude, package = "tm") # load in a tm example VCorpus mytmCorpus <- corpus(crude) summary(mytmCorpus, showmeta=TRUE) data(acq, package = "tm") summary(corpus(acq), 5, showmeta=TRUE) tmCorp <- tm::VCorpus(tm::VectorSource(data_char_ukimmig2010)) quantCorp <- corpus(tmCorp) summary(quantCorp) }#> Corpus consisting of 20 documents. #> #> Text Types Tokens Sentences datetimestamp description #> 127 62 103 5 1987-02-26 17:00:56 #> 144 238 495 19 1987-02-26 17:34:11 #> 191 47 62 4 1987-02-26 18:18:00 #> 194 55 74 5 1987-02-26 18:21:01 #> 211 67 97 4 1987-02-26 19:00:57 #> 236 256 519 22 1987-03-01 03:25:46 #> 237 252 494 22 1987-03-01 03:39:14 #> 242 116 179 8 1987-03-01 05:27:27 #> 246 193 366 16 1987-03-01 08:22:30 #> 248 201 386 16 1987-03-01 18:31:44 #> 273 203 410 14 1987-03-02 01:05:49 #> 349 72 106 4 1987-03-02 07:39:23 #> 352 75 115 4 1987-03-02 07:43:22 #> 353 77 106 5 1987-03-02 07:43:41 #> 368 76 118 5 1987-03-02 08:25:42 #> 489 98 160 6 1987-03-02 11:20:05 #> 502 126 215 8 1987-03-02 11:28:26 #> 543 54 94 5 1987-03-02 12:13:46 #> 704 153 321 11 1987-03-02 14:38:34 #> 708 40 59 3 1987-03-02 14:49:06 #> heading id language #> DIAMOND SHAMROCK (DIA) CUTS CRUDE PRICES 127 en #> OPEC MAY HAVE TO MEET TO FIRM PRICES - ANALYSTS 144 en #> TEXACO CANADA <TXC> LOWERS CRUDE POSTINGS 191 en #> MARATHON PETROLEUM REDUCES CRUDE POSTINGS 194 en #> HOUSTON OIL <HO> RESERVES STUDY COMPLETED 211 en #> KUWAIT SAYS NO PLANS FOR EMERGENCY OPEC TALKS 236 en #> INDONESIA SEEN AT CROSSROADS OVER ECONOMIC CHANGE 237 en #> SAUDI RIYAL DEPOSIT RATES REMAIN FIRM 242 en #> QATAR UNVEILS BUDGET FOR FISCAL 1987/88 246 en #> SAUDI ARABIA REITERATES COMMITMENT TO OPEC PACT 248 en #> SAUDI FEBRUARY CRUDE OUTPUT PUT AT 3.5 MLN BPD 273 en #> GULF ARAB DEPUTY OIL MINISTERS TO MEET IN BAHRAIN 349 en #> SAUDI ARABIA REITERATES COMMITMENT TO OPEC ACCORD 352 en #> KUWAIT MINISTER SAYS NO EMERGENCY OPEC TALKS SET 353 en #> PHILADELPHIA PORT CLOSED BY TANKER CRASH 368 en #> STUDY GROUP URGES INCREASED U.S. OIL RESERVES 489 en #> STUDY GROUP URGES INCREASED U.S. OIL RESERVES 502 en #> UNOCAL <UCL> UNIT CUTS CRUDE OIL POSTED PRICES 543 en #> NYMEX WILL EXPAND OFF-HOUR TRADING APRIL ONE 704 en #> ARGENTINE OIL PRODUCTION DOWN IN JANUARY 1987 708 en #> origin topics lewissplit cgisplit oldid #> Reuters-21578 XML YES TRAIN TRAINING-SET 5670 #> Reuters-21578 XML YES TRAIN TRAINING-SET 5687 #> Reuters-21578 XML YES TRAIN TRAINING-SET 5734 #> Reuters-21578 XML YES TRAIN TRAINING-SET 5737 #> Reuters-21578 XML YES TRAIN TRAINING-SET 5754 #> Reuters-21578 XML YES TRAIN TRAINING-SET 8321 #> Reuters-21578 XML YES TRAIN TRAINING-SET 8322 #> Reuters-21578 XML YES TRAIN TRAINING-SET 8327 #> Reuters-21578 XML YES TRAIN TRAINING-SET 8331 #> Reuters-21578 XML YES TRAIN TRAINING-SET 8333 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12456 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12532 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12535 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12536 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12550 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12672 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12685 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12726 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12887 #> Reuters-21578 XML YES TRAIN TRAINING-SET 12891 #> places author orgs #> usa <NA> <NA> #> usa BY TED D'AFFLISIO, Reuters opec #> canada <NA> <NA> #> usa <NA> <NA> #> usa <NA> <NA> #> kuwait ecuador <NA> opec #> indonesia usa By Jeremy Clift, Reuters worldbank #> bahrain saudi-arabia <NA> opec #> qatar <NA> <NA> #> bahrain saudi-arabia <NA> opec #> saudi-arabia uae <NA> opec #> uae bahrain saudi-arabia kuwait qatar <NA> opec #> saudi-arabia bahrain <NA> opec #> kuwait <NA> opec #> usa <NA> <NA> #> usa <NA> <NA> #> usa <NA> <NA> #> usa <NA> <NA> #> usa By BERNICE NAPACH, Reuters <NA> #> argentina <NA> <NA> #> people exchanges #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> hisham-nazer <NA> #> <NA> <NA> #> <NA> <NA> #> hisham-nazer <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> <NA> #> <NA> nymex #> <NA> <NA> #> #> Source: Converted from tm Corpus 'crude' #> Created: Thu Aug 10 12:42:43 2017 #> Notes: #> #> Corpus consisting of 50 documents, showing 5 documents. #> #> Text Types Tokens Sentences datetimestamp description #> 10 120 233 9 1987-02-26 15:18:06 #> 12 89 146 6 1987-02-26 15:19:15 #> 44 62 86 6 1987-02-26 15:49:56 #> 45 229 421 22 1987-02-26 15:51:17 #> 68 39 55 3 1987-02-26 16:08:33 #> heading id language origin #> COMPUTER TERMINAL SYSTEMS <CPML> COMPLETES SALE 10 en Reuters-21578 XML #> OHIO MATTRESS <OMT> MAY HAVE LOWER 1ST QTR NET 12 en Reuters-21578 XML #> MCLEAN'S <MII> U.S. LINES SETS ASSET TRANSFER 44 en Reuters-21578 XML #> CHEMLAWN <CHEM> RISES ON HOPES FOR HIGHER BIDS 45 en Reuters-21578 XML #> <COFAB INC> BUYS GULFEX FOR UNDISCLOSED AMOUNT 68 en Reuters-21578 XML #> topics lewissplit cgisplit oldid places author #> YES TRAIN TRAINING-SET 5553 usa <NA> #> YES TRAIN TRAINING-SET 5555 usa <NA> #> YES TRAIN TRAINING-SET 5587 usa <NA> #> YES TRAIN TRAINING-SET 5588 usa By Cal Mankowski, Reuters #> YES TRAIN TRAINING-SET 5611 usa <NA> #> #> Source: Converted from tm Corpus 'acq' #> Created: Thu Aug 10 12:42:43 2017 #> Notes: #> #> Corpus consisting of 9 documents. #> #> Text Types Tokens Sentences datetimestamp id language #> text1 1125 3280 88 2017-08-10 11:42:43 1 en #> text2 142 260 4 2017-08-10 11:42:43 2 en #> text3 251 499 15 2017-08-10 11:42:43 3 en #> text4 322 679 21 2017-08-10 11:42:43 4 en #> text5 298 683 29 2017-08-10 11:42:43 5 en #> text6 251 483 14 2017-08-10 11:42:43 6 en #> text7 77 114 5 2017-08-10 11:42:43 7 en #> text8 88 134 4 2017-08-10 11:42:43 8 en #> text9 346 723 27 2017-08-10 11:42:43 9 en #> #> Source: Converted from tm Corpus 'tmCorp' #> Created: Thu Aug 10 12:42:43 2017 #> Notes: #># construct a corpus from a data.frame mydf <- data.frame(letter_factor = factor(rep(letters[1:3], each = 2)), some_ints = 1L:6L, some_text = paste0("This is text number ", 1:6, "."), stringsAsFactors = FALSE, row.names = paste0("fromDf_", 1:6)) mydf#> letter_factor some_ints some_text #> fromDf_1 a 1 This is text number 1. #> fromDf_2 a 2 This is text number 2. #> fromDf_3 b 3 This is text number 3. #> fromDf_4 b 4 This is text number 4. #> fromDf_5 c 5 This is text number 5. #> fromDf_6 c 6 This is text number 6.summary(corpus(mydf, text_field = "some_text", metacorpus = list(source = "From a data.frame called mydf.")))#> Corpus consisting of 6 documents. #> #> Text Types Tokens Sentences letter_factor some_ints #> fromDf_1 6 6 1 a 1 #> fromDf_2 6 6 1 a 2 #> fromDf_3 6 6 1 b 3 #> fromDf_4 6 6 1 b 4 #> fromDf_5 6 6 1 c 5 #> fromDf_6 6 6 1 c 6 #> #> Source: From a data.frame called mydf. #> Created: Thu Aug 10 12:42:43 2017 #> Notes: #># construct a corpus from a kwic object mykwic <- kwic(data_corpus_inaugural, "southern") summary(corpus(mykwic))#> Corpus consisting of 28 documents. #> #> Text Types Tokens Sentences docname from to keyword context #> text1.pre 5 5 1 1797-Adams 1803 1803 southern pre #> text2.pre 4 5 1 1825-Adams 2432 2432 southern pre #> text3.pre 4 5 1 1861-Lincoln 96 96 Southern pre #> text4.pre 5 5 1 1865-Lincoln 279 279 southern pre #> text5.pre 5 5 1 1877-Hayes 376 376 Southern pre #> text6.pre 5 5 1 1877-Hayes 948 948 Southern pre #> text7.pre 5 5 1 1877-Hayes 1240 1240 Southern pre #> text8.pre 5 5 1 1881-Garfield 991 991 Southern pre #> text9.pre 4 5 1 1909-Taft 4027 4027 Southern pre #> text10.pre 5 5 1 1909-Taft 4228 4228 Southern pre #> text11.pre 5 5 1 1909-Taft 4348 4348 Southern pre #> text12.pre 5 5 1 1909-Taft 4533 4533 Southern pre #> text13.pre 5 5 1 1909-Taft 4593 4593 Southern pre #> text14.pre 5 5 1 1953-Eisenhower 1226 1226 southern pre #> text1.post 5 5 1 1797-Adams 1803 1803 southern post #> text2.post 5 5 1 1825-Adams 2432 2432 southern post #> text3.post 5 5 1 1861-Lincoln 96 96 Southern post #> text4.post 5 5 2 1865-Lincoln 279 279 southern post #> text5.post 5 5 2 1877-Hayes 376 376 Southern post #> text6.post 5 5 1 1877-Hayes 948 948 Southern post #> text7.post 5 5 1 1877-Hayes 1240 1240 Southern post #> text8.post 5 5 2 1881-Garfield 991 991 Southern post #> text9.post 5 5 2 1909-Taft 4027 4027 Southern post #> text10.post 5 5 1 1909-Taft 4228 4228 Southern post #> text11.post 5 5 1 1909-Taft 4348 4348 Southern post #> text12.post 5 5 1 1909-Taft 4533 4533 Southern post #> text13.post 5 5 1 1909-Taft 4593 4593 Southern post #> text14.post 5 5 1 1953-Eisenhower 1226 1226 southern post #> #> Source: Corpus created from kwic(x, keywords = "southern") #> Created: Thu Aug 10 12:42:43 2017 #> Notes: #>