adplus-dvertising
frame-decoration

Question

The process of forming tokens from an input stream of characters is called_____.

a.

Liberalisation

b.

Characterisation

c.

Tokenization

d.

None of the mentioned

Answer: (c).Tokenization

Engage with the Community - Add Your Comment

Confused About the Answer? Ask for Details Here.

Know the Explanation? Add it Here.

Q. The process of forming tokens from an input stream of characters is called_____.

Similar Questions

Discover Related MCQs

Q. Which grammar defines Lexical Syntax ?

Q. Which of the following statements is false?

Q. Which of the following grammar rules violate the requirements of an operator grammar ?
P, Q, R are nonterminals, and r, s, t are terminals.

1. P → Q R
2. P → Q s R
3. P → ε
4. P → Q t R r

Q. The action of parsing the source program into the proper syntactic classes is called

Q. Which of the following statement is true?

Q. Minimum hamming distance method is used for connection of

Q. Pee hole optimization is a form of

Q. The value of k, in LR(k) cannot be

Q. The cost of developing a compiler is proportional to

Q. ud-chaining is useful for

Q. An ideal compiler should

Q. which of the following class of statements usually produces no executable code when compiled ?

Q. A compiler which allows only the modified section of the source code to be re-compiled is called

Q. Storage mapping is done by

Q. A computer programe that converts the whole programe into machine language at a single time is called

Q. Compiler can check-

Q. System program such a s compiler are designed so that they are

Q. Compiler can diagnose

Q. A programmer, by mistake, writes an instruction to divide, instead of a multiply, such error can be detected by a/an

Q. Disadvantage of " Compile and GO " loading scheme is that