将图像编码器替换为swim transform
2025.5.4
将图像编码器visio transform替换为了swim transform，因为我想着visio transform是基于imageNet数据集（一般数据集）上训练的，可能不适合医学图像的编码，swim transform提取医学特征的能力也许会比visio transform好一些。
结果：
val: Epoch [69/100], Step [0/94], Loss: 0.9982 >0.73< >0.25< >0.01219<
eval metrics [ 0. 76143. 0. 0.] Acc Precision Recall F1-->> (1.0, 0.0, 0.0, 0.0)
normal (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
opacity (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
degenerative change (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
atelectases (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
atelectasis (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
scarring (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
cardiomegaly (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
calcified granuloma (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
granuloma (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pneumonia (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pleural effusion (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
sternotomy (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pleural effusions (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pulmonary emphysema (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
infiltrates (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
emphysemas (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
granulomatous disease (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
nodule (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pulmonary edema (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
diaphragm (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
emphysema (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
deformity (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
thoracic aorta (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
osteophytes (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
hiatal hernia (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
thoracic vertebrae (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
fracture (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
tortuous aorta (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
bilateral pleural effusion (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
rib fracture (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
aorta (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
edemas (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
calcinosis (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
scar (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
edema (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
copd (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pulmonary disease, chronic obstructive (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pneumothorax (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
effusion (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pleural thickening (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
pulmonary atelectasis (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
congestion (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
eventration (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
rib fractures (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
hyperinflation (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
arthritic changes (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
ribs (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
cabg (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
catheterization, central venous (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
infection (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
others (1.0, 0.0, 0.0, 0.0) [ 0. 1493. 0. 0.]
['normal normal xray chest pa and lateral xxxx year old no known xxxx left anterior chest pain.', 'xxxx.', 'none.', 'the heart is normal in size and contour.', 'the lungs are clear without evidence of infiltrate.', 'there is no pneumothorax or effusion.', 'no acute cardiopulmonary disease.']
['normal normal xray chest pa and lateral xxxx year old xxxx a a xxxx days rib pain xxxx.', 'none.', 'the.', 'the lungs is normal in size and contour.', 'no lungs are clear without evidence of focal.', 'no are no pneumothorax effusion.']
Some weights of RobertaModel were not initialized from the model checkpoint at roberta-large and are newly initialized: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
calculating scores...
computing bert embedding.
100%|██████████| 45/45 [05:19<00:00, 7.10s/it]
computing greedy matching.
100%|██████████| 24/24 [00:00<00:00, 46.20it/s]
done in 320.03 seconds, 4.67 sentences/sec
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.01s)
creating index...
index created!
tokenization...
PTBTokenizer tokenized 93914 tokens at 640299.88 tokens per second.
PTBTokenizer tokenized 80511 tokens at 836456.20 tokens per second.
setting up scorers...
computing Bleu score...
{'testlen': 69708, 'reflen': 81602, 'guess': [69708, 68215, 66722, 65229], 'correct': [50056, 32941, 24074, 18463]}
ratio: 0.8542437685350744
Bleu_1: 0.605
Bleu_2: 0.496
Bleu_3: 0.422
Bleu_4: 0.366
computing METEOR score...
METEOR: 0.307
computing Rouge score...
ROUGE_L: 0.611
computing CIDEr score...
CIDEr: 2.209
Bleu_1: 0.605
Bleu_2: 0.496
Bleu_3: 0.422
Bleu_4: 0.366
METEOR: 0.307
ROUGE_L: 0.611
CIDEr: 2.209
BERTScore_Precision: 0.911
BERTScore_Recall: 0.902
BERTScore_F1: 0.906
可以说是没什么提升（原模型的B1也能达到0.601左右），我猜想是模块之前的适配能力，因为visio transform自带一个[cls]token可以提供给MLC模块一个全局信息的特征。而swim transform没有，因此为了得到一个代表全局信息的向量给 MLC 使用（对应原文的 fp 或 sem_enc_output 的初始来源），我对所有 patch 的特征在 num_patches 维度上进行平均池化 (features.mean(dim=1, keepdim=True)），得到 sem_features。keepdim=True 保持了维度为 [B, 1, D]。这个地方与MLC模块的适配程度可能不如MLC。
因此，最后还是换回了visio transform
tag_l的重写
在源代码中，对于IU X-ray数据集，给出了以下tag标签列表，包含所有数据的标签（分类）：
tags_l = ['normal', 'opacity', 'degenerative change', 'atelectases', 'atelectasis', 'scarring',
'cardiomegaly', 'calcified granuloma', 'granuloma', 'pneumonia', 'pleural effusion', 'sternotomy',
'pleural effusions', 'pulmonary emphysema', 'infiltrates', 'emphysemas', 'granulomatous disease',
'nodule', 'pulmonary edema', 'diaphragm', 'emphysema', 'deformity', 'thoracic aorta',
'osteophytes', 'hiatal hernia', 'thoracic vertebrae', 'fracture', 'tortuous aorta',
'bilateral pleural effusion', 'rib fracture', 'aorta', 'edemas', 'calcinosis', 'scar', 'edema',
'copd', 'pulmonary disease, chronic obstructive', 'pneumothorax', 'effusion', 'pleural thickening',
'pulmonary atelectasis', 'congestion', 'eventration', 'rib fractures', 'hyperinflation',
'arthritic changes', 'ribs', 'cabg', 'catheterization, central venous', 'infection', 'others']
但我在查最原始的report信息时，发现并不能完全对上，这些标签有可能是后来人工整理的，既然和训练集（train_nt.json）中的tag不能一一对上，那后面计算focal loss的时候就会出现问题。
因此我将原report报告中Problems一列出现过的单词作为新的标签集，并去除出现频率≤3的标签，将其替换为“others”标签。最后的tags_l列表被我修改为：
tags_l = [
'abdomen', 'airspace disease', 'aorta', 'aorta, thoracic', 'arthritis',
'atherosclerosis', 'blister', 'cardiac shadow', 'cardiomegaly',
'catheters, indwelling', 'cavitation', 'consolidation', 'costophrenic angle',
'cysts', 'deformity', 'density', 'diaphragm', 'diaphragmatic eventration',
'dislocations', 'emphysema', 'epicardial fat', 'fibrosis', 'foreign bodies',
'fractures, bone', 'granuloma', 'granulomatous disease', 'heart',
'heart failure', 'hernia, hiatal', 'hyperostosis, diffuse idiopathic skeletal',
'hypertension, pulmonary', 'implanted medical device', 'infiltrate',
'kyphosis', 'lucency', 'lumbar vertebrae', 'lung', 'lung diseases, interstitial',
'lung, hyperlucent', 'markings', 'mass', 'mastectomy', 'mediastinum',
'medical device', 'nipple shadow', 'nodule', 'normal', 'opacity', 'osteophyte',
'pericardial effusion', 'pleural effusion', 'pneumonectomy', 'pneumonia',
'pneumothorax', 'pulmonary artery', 'pulmonary atelectasis', 'pulmonary congestion',
'pulmonary disease, chronic obstructive', 'pulmonary edema', 'pulmonary emphysema',
'pulmonary fibrosis', 'sclerosis', 'scoliosis', 'shift', 'shoulder', 'spinal fusion',
'spine', 'spondylosis', 'stents', 'sulcus', 'surgical instruments', 'sutures',
'thickening', 'tube, inserted', 'volume loss', 'others'
]
而相对应的训练集train_nt.json中
使得每个数据中的tag内容都能在重写后的tag_l列表中找到。
最后，在修改完tag_l列表后，代码跑出来的结果为：
val: Epoch [57/100], Step [0/47], Loss: 0.8329 >0.55< >0.26< >0.01932<
eval metrics [ 536. 37142. 208. 211.] Acc Precision Recall F1-->> (0.989, 0.7204, 0.7175, 0.7189)
normal (0.7216, 0.6947, 0.5304, 0.6015) [157. 382. 69. 139.]
opacity (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
degenerative change (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
atelectases (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
atelectasis (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
scarring (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
cardiomegaly (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
calcified granuloma (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
granuloma (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pneumonia (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pleural effusion (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
sternotomy (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pleural effusions (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pulmonary emphysema (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
infiltrates (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
emphysemas (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
granulomatous disease (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
nodule (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pulmonary edema (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
diaphragm (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
emphysema (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
deformity (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
thoracic aorta (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
osteophytes (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
hiatal hernia (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
thoracic vertebrae (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
fracture (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
tortuous aorta (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
bilateral pleural effusion (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
rib fracture (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
aorta (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
edemas (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
calcinosis (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
scar (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
edema (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
copd (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pulmonary disease, chronic obstructive (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pneumothorax (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
effusion (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pleural thickening (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pulmonary atelectasis (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
congestion (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
eventration (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
rib fractures (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
hyperinflation (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
arthritic changes (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
ribs (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
cabg (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
catheterization, central venous (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
infection (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
others (0.7175, 0.7317, 0.8404, 0.7823) [379. 157. 139. 72.]
['low lung volumes with bronchovascular crowding.', 'sequela of prior granulomatous disease.', 'otherwise lungs clear.', 'heart size normal.', 'stable severe l xxxx deformity.']
['the lung volumes are bronchovascular crowding at.', 'the of prior granulomatous disease.', 'the the are.', 'no size is.', 'no degenerative degenerative xxxx sternotomy.']
Some weights of RobertaModel were not initialized from the model checkpoint at roberta-large and are newly initialized: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
0%| | 0/19 [00:00<?, ?it/s]calculating scores...
computing bert embedding.
100%|██████████| 19/19 [00:01<00:00, 10.31it/s]
computing greedy matching.
100%|██████████| 12/12 [00:00<00:00, 67.26it/s]
done in 2.03 seconds, 367.94 sentences/sec
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.00s)
creating index...
index created!
tokenization...
PTBTokenizer tokenized 24193 tokens at 410974.50 tokens per second.
PTBTokenizer tokenized 20273 tokens at 392841.92 tokens per second.
setting up scorers...
computing Bleu score...
{'testlen': 16823, 'reflen': 20310, 'guess': [16823, 16079, 15424, 14769], 'correct': [12687, 8492, 6009, 4428]}
ratio: 0.8283111767601758
Bleu_1: 0.613
Bleu_2: 0.513
Bleu_3: 0.437
Bleu_4: 0.377
computing METEOR score...
METEOR: 0.323
computing Rouge score...
ROUGE_L: 0.561
computing CIDEr score...
CIDEr: 2.324
Bleu_1: 0.613
Bleu_2: 0.513
Bleu_3: 0.437
Bleu_4: 0.377
METEOR: 0.323
ROUGE_L: 0.561
CIDEr: 2.324
BERTScore_Precision: 0.927
BERTScore_Recall: 0.915
BERTScore_F1: 0.921
次好的：
val: Epoch [66/100], Step [0/47], Loss: 1.2255 >0.94< >0.25< >0.02784<
eval metrics [ 549. 37149. 201. 198.] Acc Precision Recall F1-->> (0.9895, 0.732, 0.7349, 0.7334)
normal (0.7282, 0.6755, 0.6047, 0.6381) [179. 365. 86. 117.]
opacity (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
degenerative change (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
atelectases (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
atelectasis (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
scarring (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
cardiomegaly (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
calcified granuloma (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
granuloma (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pneumonia (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pleural effusion (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
sternotomy (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pleural effusions (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pulmonary emphysema (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
infiltrates (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
emphysemas (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
granulomatous disease (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
nodule (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pulmonary edema (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
diaphragm (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
emphysema (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
deformity (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
thoracic aorta (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
osteophytes (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
hiatal hernia (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
thoracic vertebrae (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
fracture (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
tortuous aorta (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
bilateral pleural effusion (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
rib fracture (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
aorta (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
edemas (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
calcinosis (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
scar (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
edema (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
copd (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pulmonary disease, chronic obstructive (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pneumothorax (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
effusion (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pleural thickening (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
pulmonary atelectasis (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
congestion (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
eventration (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
rib fractures (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
hyperinflation (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
arthritic changes (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
ribs (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
cabg (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
catheterization, central venous (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
infection (1.0, 0.0, 0.0, 0.0) [ 0. 747. 0. 0.]
others (0.7376, 0.7629, 0.8204, 0.7906) [370. 181. 115. 81.]
['both lungs remain hyperexpanded.', 'no xxxx focal infiltrates.', 'a small pleural or collection is xxxx present in the right apex.', 'however it has decreased considerably since the previous examination.', 'heart size remains normal.']
['the lungs remain hyperexpanded with.', 'there pleural infiltrates infiltrates.', 'no small pleural or collection is xxxx present in the right apex.', 'heart it has decreased considerably since the previous examination.', 'no size remains normal.']
Some weights of RobertaModel were not initialized from the model checkpoint at roberta-large and are newly initialized: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
calculating scores...
computing bert embedding.
100%|██████████| 19/19 [00:01<00:00, 10.44it/s]
computing greedy matching.
100%|██████████| 12/12 [00:00<00:00, 68.02it/s]
done in 2.01 seconds, 372.39 sentences/sec
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.00s)
creating index...
index created!
tokenization...
PTBTokenizer tokenized 24193 tokens at 398011.34 tokens per second.
PTBTokenizer tokenized 20250 tokens at 350066.21 tokens per second.
setting up scorers...
computing Bleu score...
{'testlen': 16803, 'reflen': 20310, 'guess': [16803, 16058, 15403, 14748], 'correct': [12608, 8472, 6065, 4534]}
ratio: 0.8273264401772118
Bleu_1: 0.609
Bleu_2: 0.511
Bleu_3: 0.437
Bleu_4: 0.380
computing METEOR score...
METEOR: 0.323
computing Rouge score...
ROUGE_L: 0.555
computing CIDEr score...
CIDEr: 2.366
Bleu_1: 0.609
Bleu_2: 0.511
Bleu_3: 0.437
Bleu_4: 0.380
METEOR: 0.323
ROUGE_L: 0.555
CIDEr: 2.366
BERTScore_Precision: 0.928
BERTScore_Recall: 0.914
BERTScore_F1: 0.921
可以看到B1和B4都提升了一个百分点，CIDEr分数也从2.2提升到了2.3
Comments NOTHING