|
|
|
|
| LEADER |
12344nmm a2200469 u 4500 |
| 001 |
EB002067666 |
| 003 |
EBX01000000000000001207756 |
| 005 |
00000000000000.0 |
| 007 |
cr||||||||||||||||||||| |
| 008 |
220922 ||| eng |
| 050 |
|
4 |
|a Q336
|
| 100 |
1 |
|
|a Rothman, Denis
|
| 245 |
0 |
0 |
|a Transformers for natural language processing
|b build, train, and fine-tuning deep neural network architectures for NLP with Python, PyTorch, TensorFlow, BERT, and GPT-3
|c Denis Rothman ; foreword by Antonio Gulli
|
| 250 |
|
|
|a Second edition
|
| 260 |
|
|
|a [Birmingham, United Kingdom]
|b Packt Publishing
|c 2022
|
| 300 |
|
|
|a 564 pages
|b illustrations
|
| 505 |
0 |
|
|a Intro -- Copyright -- Foreword -- Contributors -- Table of Contents -- Preface -- Chapter 1: What are Transformers? -- The ecosystem of transformers -- Industry 4.0 -- Foundation models -- Is programming becoming a sub-domain of NLP? -- The future of artificial intelligence specialists -- Optimizing NLP models with transformers -- The background of transformers -- What resources should we use? -- The rise of Transformer 4.0 seamless APIs -- Choosing ready-to-use API-driven libraries -- Choosing a Transformer Model -- The role of Industry 4.0 artificial intelligence specialists -- Summary -- Questions -- References -- Chapter 2: Getting Started with the Architecture of the Transformer Model -- The rise of the Transformer: Attention is All You Need -- The encoder stack -- Input embedding -- Positional encoding -- Sublayer 1: Multi-head attention -- Sublayer 2: Feedforward network -- The decoder stack -- Output embedding and position encoding -- The attention layers -- The FFN sublayer, the post-LN, and the linear layer -- Training and performance -- Tranformer models in Hugging Face -- Summary -- Questions -- References -- Chapter 3: Fine-Tuning BERT Models -- The architecture of BERT -- The encoder stack -- Preparing the pretraining input environment -- Pretraining and fine-tuning a BERT model -- Fine-tuning BERT -- Hardware constraints -- Installing the Hugging Face PyTorch interface for BERT -- Importing the modules -- Specifying CUDA as the device for torch -- Loading the dataset -- Creating sentences, label lists, and adding BERT tokens -- Activating the BERT tokenizer -- Processing the data -- Creating attention masks -- Splitting the data into training and validation sets -- Converting all the data into torch tensors -- Selecting a batch size and creating an iterator -- BERT model configuration
|
| 505 |
0 |
|
|a Chapter 10: Semantic Role Labeling with BERT-Based Transformers -- Getting started with SRL -- Defining semantic role labeling -- Visualizing SRL -- Running a pretrained BERT-based model -- The architecture of the BERT-based model -- Setting up the BERT SRL environment -- SRL experiments with the BERT-based model -- Basic samples -- Sample 1 -- Sample 2 -- Sample 3 -- Difficult samples -- Sample 4 -- Sample 5 -- Sample 6 -- Questioning the scope of SRL -- The limit of predicate analysis -- Redefining SRL -- Summary -- Questions -- References -- Chapter 11: Let Your Data Do the Talking: Story, Questions, and Answers -- Methodology -- Transformers and methods -- Method 0: Trial and error -- Method 1: NER first -- Using NER to find questions -- Location entity questions -- Person entity questions -- Method 2: SRL first -- Question-answering with ELECTRA -- Project management constraints -- Using SRL to find questions -- Next steps -- Exploring Haystack with a RoBERTa model -- Exploring Q& -- A with a GTP-3 engine -- Summary -- Questions -- References -- Chapter 12: Detecting Customer Emotions to Make Predictions -- Getting started: Sentiment analysis transformers -- The Stanford Sentiment Treebank (SST) -- Sentiment analysis with RoBERTa-large -- Predicting customer behavior with sentiment analysis -- Sentiment analysis with DistilBERT -- Sentiment analysis with Hugging Face's models' list -- DistilBERT for SST -- MiniLM-L12-H384-uncased -- RoBERTa-large-mnli -- BERT-base multilingual model -- Sentiment analysis with GPT-3 -- Some Pragmatic I4.0 thinking before we leave -- Investigating with SRL -- Investigating with Hugging Face -- Investigating with the GPT-3 playground -- GPT-3 code -- Summary -- Questions -- References -- Chapter 13: Analyzing Fake News with Transformers -- Emotional reactions to fake news
|
| 505 |
0 |
|
|a NLP tasks and examples -- Comparing the output of GPT-2 and GPT-3 -- Fine-tuning GPT-3 -- Preparing the data -- Step 1: Installing OpenAI -- Step 2: Entering the API key -- Step 3: Activating OpenAI's data preparation module -- Fine-tuning GPT-3 -- Step 4: Creating an OS environment -- Step 5: Fine-tuning OpenAI's Ada engine -- Step 6: Interacting with the fine-tuned model -- The role of an Industry 4.0 AI specialist -- Initial conclusions -- Summary -- Questions -- References -- Chapter 8: Applying Transformers to Legal and Financial Documents for AI Text Summarization -- Designing a universal text-to-text model -- The rise of text-to-text transformer models -- A prefix instead of task-specific formats -- The T5 model -- Text summarization with T5 -- Hugging Face -- Hugging Face transformer resources -- Initializing the T5-large transformer model -- Getting started with T5 -- Exploring the architecture of the T5 model -- Summarizing documents with T5-large -- Creating a summarization function -- A general topic sample -- The Bill of Rights sample -- A corporate law sample -- Summarization with GPT-3 -- Summary -- Questions -- References -- Chapter 9: Matching Tokenizers and Datasets -- Matching datasets and tokenizers -- Best practices -- Step 1: Preprocessing -- Step 2: Quality control -- Continuous human quality control -- Word2Vec tokenization -- Case 0: Words in the dataset and the dictionary -- Case 1: Words not in the dataset or the dictionary -- Case 2: Noisy relationships -- Case 3: Words in the text but not in the dictionary -- Case 4: Rare words -- Case 5: Replacing rare words -- Case 6: Entailment -- Standard NLP tasks with specific vocabulary -- Generating unconditional samples with GPT-2 -- Generating trained conditional samples -- Controlling tokenized data -- Exploring the scope of GPT-3 -- Summary -- Questions -- References
|
| 505 |
0 |
|
|a Loading the Hugging Face BERT uncased base model -- Optimizer grouped parameters -- The hyperparameters for the training loop -- The training loop -- Training evaluation -- Predicting and evaluating using the holdout dataset -- Evaluating using the Matthews Correlation Coefficient -- The scores of individual batches -- Matthews evaluation for the whole dataset -- Summary -- Questions -- References -- Chapter 4: Pretraining a RoBERTa Model from Scratch -- Training a tokenizer and pretraining a transformer -- Building KantaiBERT from scratch -- Step 1: Loading the dataset -- Step 2: Installing Hugging Face transformers -- Step 3: Training a tokenizer -- Step 4: Saving the files to disk -- Step 5: Loading the trained tokenizer files -- Step 6: Checking resource constraints: GPU and CUDA -- Step 7: Defining the configuration of the model -- Step 8: Reloading the tokenizer in transformers -- Step 9: Initializing a model from scratch -- Exploring the parameters -- Step 10: Building the dataset -- Step 11: Defining a data collator -- Step 12: Initializing the trainer -- Step 13: Pretraining the model -- Step 14: Saving the final model (+tokenizer + config) to disk -- Step 15: Language modeling with FillMaskPipeline -- Next steps -- Summary -- Questions -- References -- Chapter 5: Downstream NLP Tasks with Transformers -- Transduction and the inductive inheritance of transformers -- The human intelligence stack -- The machine intelligence stack -- Transformer performances versus Human Baselines -- Evaluating models with metrics -- Accuracy score -- F1-score -- Matthews Correlation Coefficient (MCC) -- Benchmark tasks and datasets -- From GLUE to SuperGLUE -- Introducing higher Human Baselines standards -- The SuperGLUE evaluation process -- Defining the SuperGLUE benchmark tasks -- BoolQ -- Commitment Bank (CB)
|
| 505 |
0 |
|
|a Multi-Sentence Reading Comprehension (MultiRC) -- Reading Comprehension with Commonsense Reasoning Dataset (ReCoRD) -- Recognizing Textual Entailment (RTE) -- Words in Context (WiC) -- The Winograd schema challenge (WSC) -- Running downstream tasks -- The Corpus of Linguistic Acceptability (CoLA) -- Stanford Sentiment TreeBank (SST-2) -- Microsoft Research Paraphrase Corpus (MRPC) -- Winograd schemas -- Summary -- Questions -- References -- Chapter 6: Machine Translation with the Transformer -- Defining machine translation -- Human transductions and translations -- Machine transductions and translations -- Preprocessing a WMT dataset -- Preprocessing the raw data -- Finalizing the preprocessing of the datasets -- Evaluating machine translation with BLEU -- Geometric evaluations -- Applying a smoothing technique -- Chencherry smoothing -- Translation with Google Translate -- Translations with Trax -- Installing Trax -- Creating the original Transformer model -- Initializing the model using pretrained weights -- Tokenizing a sentence -- Decoding from the Transformer -- De-tokenizing and displaying the translation -- Summary -- Questions -- References -- Chapter 7: The Rise of Suprahuman Transformers with GPT-3 Engines -- Suprahuman NLP with GPT-3 transformer models -- The architecture of OpenAI GPT transformer models -- The rise of billion-parameter transformer models -- The increasing size of transformer models -- Context size and maximum path length -- From fine-tuning to zero-shot models -- Stacking decoder layers -- GPT-3 engines -- Generic text completion with GPT-2 -- Step 9: Interacting with GPT-2 -- Training a custom GPT-2 language model -- Step 12: Interactive context and completion examples -- Running OpenAI GPT-3 tasks -- Running NLP tasks online -- Getting started with GPT-3 engines -- Running our first NLP task with GPT-3
|
| 653 |
|
|
|a Artificial intelligence / Data processing / http://id.loc.gov/authorities/subjects/sh85008182
|
| 653 |
|
|
|a Cloud computing / fast
|
| 653 |
|
|
|a Intelligence artificielle / Logiciels
|
| 653 |
|
|
|a Python (Computer program language) / fast
|
| 653 |
|
|
|a Infonuagique
|
| 653 |
|
|
|a Python (Computer program language) / http://id.loc.gov/authorities/subjects/sh96008834
|
| 653 |
|
|
|a Artificial intelligence / Computer programs / http://id.loc.gov/authorities/subjects/sh85008181
|
| 653 |
|
|
|a Cloud computing / http://id.loc.gov/authorities/subjects/sh2008004883
|
| 653 |
|
|
|a Artificial intelligence / Data processing / fast
|
| 653 |
|
|
|a Intelligence artificielle / Informatique
|
| 653 |
|
|
|a Python (Langage de programmation)
|
| 653 |
|
|
|a Artificial intelligence / Computer programs / fast
|
| 700 |
1 |
|
|a Gulli, Antonio
|e writer of forewrod
|
| 041 |
0 |
7 |
|a eng
|2 ISO 639-2
|
| 989 |
|
|
|b OREILLY
|a O'Reilly
|
| 490 |
0 |
|
|a Expert insight
|
| 500 |
|
|
|a Includes index
|
| 776 |
|
|
|z 9781803247335
|
| 856 |
4 |
0 |
|u https://learning.oreilly.com/library/view/~/9781803247335/?ar
|x Verlag
|3 Volltext
|
| 082 |
0 |
|
|a 006.3
|
| 520 |
|
|
|a Transformers are a game-changer for natural language understanding (NLU) and have become one of the pillars of artificial intelligence. Transformers for Natural Language Processing, 2nd Edition, investigates deep learning for machine translations, speech-to-text, text-to-speech, language modeling, question-answering, and many more NLP domains with transformers. An Industry 4.0 AI specialist needs to be adaptable; knowing just one NLP platform is not enough anymore. Different platforms have different benefits depending on the application, whether it's cost, flexibility, ease of implementation, results, or performance. In this book, we analyze numerous use cases with Hugging Face, Google Trax, OpenAI, and AllenNLP. This book takes transformers' capabilities further by combining multiple NLP techniques, such as sentiment analysis, named entity recognition, and semantic role labeling, to analyze complex use cases, such as dissecting fake news on Twitter. Also, see how transformers can create code using just a brief description. By the end of this NLP book, you will understand transformers from a cognitive science perspective and be proficient in applying pretrained transformer models to various datasets
|