Lựa chọn mô hình | BERT BiLSTM LSTM GRU CNN

Cấu trúc:

Model: "BERT"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to
==================================================================================================
 input_ids (InputLayer)         [(None, 256)]        0           []

 attention_mask (InputLayer)    [(None, 256)]        0           []

 tf_roberta_model (TFRobertaMod  TFBaseModelOutputWi  134998272  ['input_ids[0][0]',
 el)                            thPoolingAndCrossAt               'attention_mask[0][0]']
                                tentions(last_hidde
                                n_state=(None, 256,
                                 768),
                                 pooler_output=(Non
                                e, 768),
                                 past_key_values=No
                                ne, hidden_states=N
                                one, attentions=Non
                                e, cross_attentions
                                =None)

 flatten (Flatten)              (None, 196608)       0           ['tf_roberta_model[0][0]']

 batch_normalization (BatchNorm  (None, 196608)      786432      ['flatten[0][0]']
 alization)

 dense (Dense)                  (None, 64)           12582976    ['batch_normalization[0][0]']

 dropout_37 (Dropout)           (None, 64)           0           ['dense[0][0]']

 outputs (Dense)                (None, 2)            130         ['dropout_37[0][0]']

==================================================================================================
Total params: 148,367,810
Trainable params: 12,976,322
Non-trainable params: 135,391,488
__________________________________________________________________________________________________
                

N = 421	Dự đoán là tin thật	Dự đoán là tin giả
Tin thật (204)	193	10
Tin giả (217)	13	205

N = 421
{'loss': 2.30495023727417, 'accuracy': 0.9360189437866211}
                

Model: "BiLSTM"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 embedding (Embedding)       (None, 512, 128)          1280000

 bidirectional (Bidirectiona  (None, 256)              263168
 l)

 dense (Dense)               (None, 128)               32896

 dense_1 (Dense)             (None, 64)                8256

 dense_2 (Dense)             (None, 1)                 65

=================================================================
Total params: 1,584,385
Trainable params: 1,584,385
Non-trainable params: 0
_________________________________________________________________
                

N = 421	Dự đoán là tin thật	Dự đoán là tin giả
Tin thật (204)	185	21
Tin giả (217)	17	198

N = 421
{'loss': 0.26830601692199707, 'accuracy': 0.9099525809288025}
                

Model: "LSTM"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 embedding (Embedding)       (None, 512, 128)          1280000

 lstm (LSTM)                 (None, 128)               131584

 dense (Dense)               (None, 128)               16512

 dense_1 (Dense)             (None, 64)                8256

 dense_2 (Dense)             (None, 1)                 65

=================================================================
Total params: 1,436,417
Trainable params: 1,436,417
Non-trainable params: 0
_________________________________________________________________
                

N = 421	Dự đoán là tin thật	Dự đoán là tin giả
Tin thật (204)	168	43
Tin giả (217)	35	175

N = 421
{'loss': 0.44977718591690063, 'accuracy': 0.8151658773422241}
                

Model: "GRU"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 embedding (Embedding)       (None, 512, 128)          1280000

 gru (GRU)                   (None, 128)               99072

 dense (Dense)               (None, 128)               16512

 dense_1 (Dense)             (None, 64)                8256

 dense_2 (Dense)             (None, 1)                 65

=================================================================
Total params: 1,403,905
Trainable params: 1,403,905
Non-trainable params: 0
_________________________________________________________________
                

N = 421	Dự đoán là tin thật	Dự đoán là tin giả
Tin thật (204)	154	52
Tin giả (217)	44	171

N = 421
{'loss': 0.4920154809951782, 'accuracy': 0.7725118398666382}
                

Model: "CNN"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 embedding (Embedding)       (None, 512, 128)          1280000

 conv1d (Conv1D)             (None, 508, 128)          82048

 flatten (Flatten)           (None, 65024)             0

 dense (Dense)               (None, 128)               8323200

 dense_1 (Dense)             (None, 64)                8256

 dense_2 (Dense)             (None, 1)                 65

=================================================================
Total params: 9,693,569
Trainable params: 9,693,569
Non-trainable params: 0
_________________________________________________________________
                

N = 421	Dự đoán là tin thật	Dự đoán là tin giả
Tin thật (204)	137	67
Tin giả (217)	56	161

N = 421
{'loss': 0.5959132313728333, 'accuracy': 0.7085307836532593}