Given a string, a "token" is any sequence of non blank characters
delimited by the beginning of a line, the end of a line (carriage return),
a tab, or a space. For example, given the line "Ann get your gun ",
the tokens are "Ann", "get", "your", and "gun".
Your program will consist of a single java file, Tokens.java,
containing the class
Tokens with a single static method, main.
The program uses the Scanner class to process the input.
The main method prompts the user for
a sentence then prints, one per line, the tokens in that sentence.
It keeps prompting the user and printing tokens until the
user enters an empty line (one consisting of only the carriage return).
At that point it prints out the total number of sentences read, of tokens
read, of characters read, the length of the longest token, and the number of
times that tokens of that length were read. Then it ends.
Here is a possible interaction with the program:
Enter a sentence:
Roses are red violets blue
Roses
are
red
violets
blue
Enter a sentence:
234 sw{}b 2 showers
234
sw{}b
2
showers
Enter a sentence:
The number of lines was 2
The number of words was 9
The number of characters was 38
The longest token was of length 7
There were 2 tokens of maximal length
FOR THE AMBITIOUS (2 extra points): Before terminating the program will print out all the non-empty substrings of the longest string.
Be sure to document your code and send it to the TA