Count the total number of number tokens and sentences.

textstat_summary(x, cache = TRUE, ...)

Arguments

x

corpus to be summarized

cache

if TRUE, use internal cache from the second time. Not available on Solaris.

...

additional arguments passed through to dfm()

Details

Count the total number of characters, tokens and sentences as well as special tokens such as numbers, punctuation marks, symbols, tags and emojis.

  • chars = number of characters; equal to nchar()

  • sents = number of sentences; equal ntoken(tokens(x), what = "sentence")

  • tokens = number of tokens; equal to ntoken()

  • types = number of unique tokens; equal to ntype()

  • puncts = number of punctuation marks (^\p{P}+$)

  • numbers = number of numeric tokens (^\p{Sc}{0,1}\p{N}+([.,]*\p{N})*\p{Sc}{0,1}$)

  • symbols = number of symbols (^\p{S}$)

  • tags = number of tags; sum of pattern_username and pattern_hashtag in quanteda_options()

  • emojis = number of emojis (^\p{Emoji_Presentation}+$)

Examples

corp <- data_corpus_inaugural textstat_summary(corp, cache = TRUE)
#> document chars sents tokens types puncts numbers symbols urls tags #> 1 1789-Washington 8619 23 1537 603 107 0 0 0 0 #> 2 1793-Washington 790 4 147 95 12 0 0 0 0 #> 3 1797-Adams 13877 37 2577 801 259 0 0 0 0 #> 4 1801-Jefferson 10140 41 1923 687 197 0 0 0 0 #> 5 1805-Jefferson 12908 45 2380 781 214 0 0 0 0 #> 6 1809-Madison 7002 21 1261 524 86 0 0 0 0 #> 7 1813-Madison 7156 33 1302 523 92 0 0 0 0 #> 8 1817-Monroe 19889 121 3677 986 307 0 0 0 0 #> 9 1821-Monroe 26325 131 4886 1207 414 11 2 0 0 #> 10 1825-Adams 17745 74 3147 968 232 0 0 0 0 #> 11 1829-Jackson 6818 25 1208 504 80 0 0 0 0 #> 12 1833-Jackson 7059 29 1267 479 91 0 0 0 0 #> 13 1837-VanBuren 23423 95 4158 1264 319 0 0 0 0 #> 14 1841-Harrison 49718 210 9123 1818 677 0 0 0 0 #> 15 1845-Polk 28722 153 5186 1268 383 1 0 0 0 #> 16 1849-Taylor 6605 22 1178 485 90 0 0 0 0 #> 17 1853-Pierce 20085 104 3636 1121 303 1 0 0 0 #> 18 1857-Buchanan 16817 89 3083 900 259 0 0 0 0 #> 19 1861-Lincoln 21023 135 3999 1018 365 4 0 0 0 #> 20 1865-Lincoln 3926 26 775 343 76 0 0 0 0 #> 21 1869-Grant 6508 40 1229 470 102 0 0 0 0 #> 22 1873-Grant 7736 43 1472 529 134 2 0 0 0 #> 23 1877-Hayes 14946 59 2707 808 223 2 0 0 0 #> 24 1881-Garfield 17769 111 3209 981 234 4 0 0 0 #> 25 1885-Cleveland 10148 44 1816 650 133 0 0 0 0 #> 26 1889-Harrison 26179 157 4721 1308 335 3 0 0 0 #> 27 1893-Cleveland 12348 58 2125 800 111 0 0 0 0 #> 28 1897-McKinley 23666 130 4353 1193 388 5 0 0 0 #> 29 1901-McKinley 13407 100 2437 822 224 8 1 0 0 #> 30 1905-Roosevelt 5567 33 1079 387 95 0 0 0 0 #> 31 1909-Taft 32164 158 5821 1383 391 2 1 0 0 #> 32 1913-Wilson 9562 68 1882 634 183 0 0 0 0 #> 33 1917-Wilson 8398 59 1652 529 123 0 0 0 0 #> 34 1921-Harding 20299 148 3719 1127 392 0 0 0 0 #> 35 1925-Coolidge 23948 196 4440 1163 384 1 0 0 0 #> 36 1929-Hoover 21768 158 3860 1002 302 0 0 0 0 #> 37 1933-Roosevelt 10907 85 2057 712 177 0 0 0 0 #> 38 1937-Roosevelt 10614 96 1989 691 181 2 0 0 0 #> 39 1941-Roosevelt 7595 68 1519 501 178 4 0 0 0 #> 40 1945-Roosevelt 3052 27 633 265 76 2 0 0 0 #> 41 1949-Truman 13687 116 2504 745 232 1 0 0 0 #> 42 1953-Eisenhower 13968 119 2743 865 289 9 0 0 0 #> 43 1957-Eisenhower 9213 92 1907 591 248 0 0 0 0 #> 44 1961-Kennedy 7642 52 1541 542 175 2 0 0 0 #> 45 1965-Johnson 8205 93 1710 535 221 3 0 0 0 #> 46 1969-Nixon 11644 103 2416 714 292 0 0 0 0 #> 47 1973-Nixon 10007 68 1995 515 193 1 0 0 0 #> 48 1977-Carter 6878 52 1369 501 145 3 0 0 0 #> 49 1981-Reagan 13743 129 2780 850 348 1 0 0 0 #> 50 1985-Reagan 14572 123 2909 876 345 11 0 0 0 #> 51 1989-Bush 12529 141 2673 756 356 2 0 0 0 #> 52 1993-Clinton 9113 81 1833 605 235 0 0 0 0 #> 53 1997-Clinton 12262 111 2436 726 279 0 0 0 0 #> 54 2001-Bush 9054 97 1806 592 222 1 0 0 0 #> 55 2005-Bush 11923 99 2312 734 241 0 0 0 0 #> 56 2009-Obama 13460 110 2689 900 299 0 0 0 0 #> 57 2013-Obama 11917 88 2317 786 220 5 0 0 0 #> 58 2017-Trump 8433 88 1660 547 215 2 0 0 0 #> emojis #> 1 0 #> 2 0 #> 3 0 #> 4 0 #> 5 0 #> 6 0 #> 7 0 #> 8 0 #> 9 0 #> 10 0 #> 11 0 #> 12 0 #> 13 0 #> 14 0 #> 15 0 #> 16 0 #> 17 0 #> 18 0 #> 19 0 #> 20 0 #> 21 0 #> 22 0 #> 23 0 #> 24 0 #> 25 0 #> 26 0 #> 27 0 #> 28 0 #> 29 0 #> 30 0 #> 31 0 #> 32 0 #> 33 0 #> 34 0 #> 35 0 #> 36 0 #> 37 0 #> 38 0 #> 39 0 #> 40 0 #> 41 0 #> 42 0 #> 43 0 #> 44 0 #> 45 0 #> 46 0 #> 47 0 #> 48 0 #> 49 0 #> 50 0 #> 51 0 #> 52 0 #> 53 0 #> 54 0 #> 55 0 #> 56 0 #> 57 0 #> 58 0
toks <- tokens(corp) textstat_summary(toks, cache = TRUE)
#> document chars sents tokens types puncts numbers symbols urls tags #> 1 1789-Washington NA NA 1537 603 107 0 0 0 0 #> 2 1793-Washington NA NA 147 95 12 0 0 0 0 #> 3 1797-Adams NA NA 2577 801 259 0 0 0 0 #> 4 1801-Jefferson NA NA 1923 687 197 0 0 0 0 #> 5 1805-Jefferson NA NA 2380 781 214 0 0 0 0 #> 6 1809-Madison NA NA 1261 524 86 0 0 0 0 #> 7 1813-Madison NA NA 1302 523 92 0 0 0 0 #> 8 1817-Monroe NA NA 3677 986 307 0 0 0 0 #> 9 1821-Monroe NA NA 4886 1207 414 11 2 0 0 #> 10 1825-Adams NA NA 3147 968 232 0 0 0 0 #> 11 1829-Jackson NA NA 1208 504 80 0 0 0 0 #> 12 1833-Jackson NA NA 1267 479 91 0 0 0 0 #> 13 1837-VanBuren NA NA 4158 1264 319 0 0 0 0 #> 14 1841-Harrison NA NA 9123 1818 677 0 0 0 0 #> 15 1845-Polk NA NA 5186 1268 383 1 0 0 0 #> 16 1849-Taylor NA NA 1178 485 90 0 0 0 0 #> 17 1853-Pierce NA NA 3636 1121 303 1 0 0 0 #> 18 1857-Buchanan NA NA 3083 900 259 0 0 0 0 #> 19 1861-Lincoln NA NA 3999 1018 365 4 0 0 0 #> 20 1865-Lincoln NA NA 775 343 76 0 0 0 0 #> 21 1869-Grant NA NA 1229 470 102 0 0 0 0 #> 22 1873-Grant NA NA 1472 529 134 2 0 0 0 #> 23 1877-Hayes NA NA 2707 808 223 2 0 0 0 #> 24 1881-Garfield NA NA 3209 981 234 4 0 0 0 #> 25 1885-Cleveland NA NA 1816 650 133 0 0 0 0 #> 26 1889-Harrison NA NA 4721 1308 335 3 0 0 0 #> 27 1893-Cleveland NA NA 2125 800 111 0 0 0 0 #> 28 1897-McKinley NA NA 4353 1193 388 5 0 0 0 #> 29 1901-McKinley NA NA 2437 822 224 8 1 0 0 #> 30 1905-Roosevelt NA NA 1079 387 95 0 0 0 0 #> 31 1909-Taft NA NA 5821 1383 391 2 1 0 0 #> 32 1913-Wilson NA NA 1882 634 183 0 0 0 0 #> 33 1917-Wilson NA NA 1652 529 123 0 0 0 0 #> 34 1921-Harding NA NA 3719 1127 392 0 0 0 0 #> 35 1925-Coolidge NA NA 4440 1163 384 1 0 0 0 #> 36 1929-Hoover NA NA 3860 1002 302 0 0 0 0 #> 37 1933-Roosevelt NA NA 2057 712 177 0 0 0 0 #> 38 1937-Roosevelt NA NA 1989 691 181 2 0 0 0 #> 39 1941-Roosevelt NA NA 1519 501 178 4 0 0 0 #> 40 1945-Roosevelt NA NA 633 265 76 2 0 0 0 #> 41 1949-Truman NA NA 2504 745 232 1 0 0 0 #> 42 1953-Eisenhower NA NA 2743 865 289 9 0 0 0 #> 43 1957-Eisenhower NA NA 1907 591 248 0 0 0 0 #> 44 1961-Kennedy NA NA 1541 542 175 2 0 0 0 #> 45 1965-Johnson NA NA 1710 535 221 3 0 0 0 #> 46 1969-Nixon NA NA 2416 714 292 0 0 0 0 #> 47 1973-Nixon NA NA 1995 515 193 1 0 0 0 #> 48 1977-Carter NA NA 1369 501 145 3 0 0 0 #> 49 1981-Reagan NA NA 2780 850 348 1 0 0 0 #> 50 1985-Reagan NA NA 2909 876 345 11 0 0 0 #> 51 1989-Bush NA NA 2673 756 356 2 0 0 0 #> 52 1993-Clinton NA NA 1833 605 235 0 0 0 0 #> 53 1997-Clinton NA NA 2436 726 279 0 0 0 0 #> 54 2001-Bush NA NA 1806 592 222 1 0 0 0 #> 55 2005-Bush NA NA 2312 734 241 0 0 0 0 #> 56 2009-Obama NA NA 2689 900 299 0 0 0 0 #> 57 2013-Obama NA NA 2317 786 220 5 0 0 0 #> 58 2017-Trump NA NA 1660 547 215 2 0 0 0 #> emojis #> 1 0 #> 2 0 #> 3 0 #> 4 0 #> 5 0 #> 6 0 #> 7 0 #> 8 0 #> 9 0 #> 10 0 #> 11 0 #> 12 0 #> 13 0 #> 14 0 #> 15 0 #> 16 0 #> 17 0 #> 18 0 #> 19 0 #> 20 0 #> 21 0 #> 22 0 #> 23 0 #> 24 0 #> 25 0 #> 26 0 #> 27 0 #> 28 0 #> 29 0 #> 30 0 #> 31 0 #> 32 0 #> 33 0 #> 34 0 #> 35 0 #> 36 0 #> 37 0 #> 38 0 #> 39 0 #> 40 0 #> 41 0 #> 42 0 #> 43 0 #> 44 0 #> 45 0 #> 46 0 #> 47 0 #> 48 0 #> 49 0 #> 50 0 #> 51 0 #> 52 0 #> 53 0 #> 54 0 #> 55 0 #> 56 0 #> 57 0 #> 58 0
dfmat <- dfm(toks) textstat_summary(dfmat, cache = TRUE)
#> document chars sents tokens types puncts numbers symbols urls tags #> 1 1789-Washington NA NA 1537 603 107 0 0 0 0 #> 2 1793-Washington NA NA 147 95 12 0 0 0 0 #> 3 1797-Adams NA NA 2577 801 259 0 0 0 0 #> 4 1801-Jefferson NA NA 1923 687 197 0 0 0 0 #> 5 1805-Jefferson NA NA 2380 781 214 0 0 0 0 #> 6 1809-Madison NA NA 1261 524 86 0 0 0 0 #> 7 1813-Madison NA NA 1302 523 92 0 0 0 0 #> 8 1817-Monroe NA NA 3677 986 307 0 0 0 0 #> 9 1821-Monroe NA NA 4886 1207 414 11 2 0 0 #> 10 1825-Adams NA NA 3147 968 232 0 0 0 0 #> 11 1829-Jackson NA NA 1208 504 80 0 0 0 0 #> 12 1833-Jackson NA NA 1267 479 91 0 0 0 0 #> 13 1837-VanBuren NA NA 4158 1264 319 0 0 0 0 #> 14 1841-Harrison NA NA 9123 1818 677 0 0 0 0 #> 15 1845-Polk NA NA 5186 1268 383 1 0 0 0 #> 16 1849-Taylor NA NA 1178 485 90 0 0 0 0 #> 17 1853-Pierce NA NA 3636 1121 303 1 0 0 0 #> 18 1857-Buchanan NA NA 3083 900 259 0 0 0 0 #> 19 1861-Lincoln NA NA 3999 1018 365 4 0 0 0 #> 20 1865-Lincoln NA NA 775 343 76 0 0 0 0 #> 21 1869-Grant NA NA 1229 470 102 0 0 0 0 #> 22 1873-Grant NA NA 1472 529 134 2 0 0 0 #> 23 1877-Hayes NA NA 2707 808 223 2 0 0 0 #> 24 1881-Garfield NA NA 3209 981 234 4 0 0 0 #> 25 1885-Cleveland NA NA 1816 650 133 0 0 0 0 #> 26 1889-Harrison NA NA 4721 1308 335 3 0 0 0 #> 27 1893-Cleveland NA NA 2125 800 111 0 0 0 0 #> 28 1897-McKinley NA NA 4353 1193 388 5 0 0 0 #> 29 1901-McKinley NA NA 2437 822 224 8 1 0 0 #> 30 1905-Roosevelt NA NA 1079 387 95 0 0 0 0 #> 31 1909-Taft NA NA 5821 1383 391 2 1 0 0 #> 32 1913-Wilson NA NA 1882 634 183 0 0 0 0 #> 33 1917-Wilson NA NA 1652 529 123 0 0 0 0 #> 34 1921-Harding NA NA 3719 1127 392 0 0 0 0 #> 35 1925-Coolidge NA NA 4440 1163 384 1 0 0 0 #> 36 1929-Hoover NA NA 3860 1002 302 0 0 0 0 #> 37 1933-Roosevelt NA NA 2057 712 177 0 0 0 0 #> 38 1937-Roosevelt NA NA 1989 691 181 2 0 0 0 #> 39 1941-Roosevelt NA NA 1519 501 178 4 0 0 0 #> 40 1945-Roosevelt NA NA 633 265 76 2 0 0 0 #> 41 1949-Truman NA NA 2504 745 232 1 0 0 0 #> 42 1953-Eisenhower NA NA 2743 865 289 9 0 0 0 #> 43 1957-Eisenhower NA NA 1907 591 248 0 0 0 0 #> 44 1961-Kennedy NA NA 1541 542 175 2 0 0 0 #> 45 1965-Johnson NA NA 1710 535 221 3 0 0 0 #> 46 1969-Nixon NA NA 2416 714 292 0 0 0 0 #> 47 1973-Nixon NA NA 1995 515 193 1 0 0 0 #> 48 1977-Carter NA NA 1369 501 145 3 0 0 0 #> 49 1981-Reagan NA NA 2780 850 348 1 0 0 0 #> 50 1985-Reagan NA NA 2909 876 345 11 0 0 0 #> 51 1989-Bush NA NA 2673 756 356 2 0 0 0 #> 52 1993-Clinton NA NA 1833 605 235 0 0 0 0 #> 53 1997-Clinton NA NA 2436 726 279 0 0 0 0 #> 54 2001-Bush NA NA 1806 592 222 1 0 0 0 #> 55 2005-Bush NA NA 2312 734 241 0 0 0 0 #> 56 2009-Obama NA NA 2689 900 299 0 0 0 0 #> 57 2013-Obama NA NA 2317 786 220 5 0 0 0 #> 58 2017-Trump NA NA 1660 547 215 2 0 0 0 #> emojis #> 1 0 #> 2 0 #> 3 0 #> 4 0 #> 5 0 #> 6 0 #> 7 0 #> 8 0 #> 9 0 #> 10 0 #> 11 0 #> 12 0 #> 13 0 #> 14 0 #> 15 0 #> 16 0 #> 17 0 #> 18 0 #> 19 0 #> 20 0 #> 21 0 #> 22 0 #> 23 0 #> 24 0 #> 25 0 #> 26 0 #> 27 0 #> 28 0 #> 29 0 #> 30 0 #> 31 0 #> 32 0 #> 33 0 #> 34 0 #> 35 0 #> 36 0 #> 37 0 #> 38 0 #> 39 0 #> 40 0 #> 41 0 #> 42 0 #> 43 0 #> 44 0 #> 45 0 #> 46 0 #> 47 0 #> 48 0 #> 49 0 #> 50 0 #> 51 0 #> 52 0 #> 53 0 #> 54 0 #> 55 0 #> 56 0 #> 57 0 #> 58 0