MASS/MASS.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# 基于MindSpore框架的MASS案例实现\n",
    "## 1 模型简介\n",
    "微软亚洲研究院于2019在ICML发表《MASS: Masked Sequence to Sequence Pre-training for Language Generation》，其借鑑了Bert的Masked Language Model预训练任务，提出了MAsked Sequence to Sequence Pre-training（MASS）模型，为自然语言生成任务联合预训练编码器和解码器。\n",
    "\n",
    "MASS的编码器-解码器结构示例，图中“_”表示被屏蔽的词。\n",
    "\n",
    "<div align=center>\n",
    "<img src='https://i.imgur.com/Jvhm0Dx.png' width='600px'>\n",
    "</div>\n",
    "\n",
    "编码器： 以被随机屏蔽掉连续片段的句子作为输入，BERT的做法是随机屏蔽掉15%的词，而MASS为了解决编码与解码之间的平衡，做法为屏蔽掉句子总长50%的片段。模型中使用特殊符号 $[\\mathbb M]$ 替换连续的单词来屏蔽片段，起始位置是随机的，且被选中的token有80%的概率是正常的 $[\\mathbb M]$ token，10%的概率是被随机token替换，10%的概率保持原来的token。以上图为例，其中输入序列有8个单词，片段 $x_3-x_6$ 被屏蔽掉。\n",
    "\n",
    "解码器：输入为与编码器同样的序列，但是会屏蔽掉剩馀的词，然后解码器只预测编码器端屏蔽掉的词。以上图为例，只给定 $x_3x_4x_5$ 作为位置 4 - 6 的解码器输入，解码器会将  $[\\mathbb M]$ 作为其他位置的输入（屏蔽了位置 1 − 3 和 7 − 8）。为了减少内存和计算成本，被屏蔽的token会被移除，未屏蔽token的位置编码不变（如果前两个标记被屏蔽并移除，第三个标记的位置仍然是 2 而不是 0)。通过这种方式，可以获得相似的准确度，并在解码器中减少 50% 的计算量。\n",
    "\n",
    "```\n",
    "encoder input (source): [x1, x2, x3, x4, x5, x6, x7, x8, </eos>]\n",
    "masked encoder input:   [x1, x2, x3,  _,  _,  _, x7, x8, </eos>]\n",
    "decoder input:          [  -, x3, x4, x5]\n",
    "                          |   |   |   |\n",
    "                          V   V   V   V\n",
    "decoder output:         [x3, x4, x5, x6]\n",
    "```\n",
    "\n",
    "MASS预训练有以下几大优势：\n",
    "\n",
    "(1) 编码器被强制去抽取未被屏蔽掉的词的含义，可以提升编码器理解源序列文本的能力。<br>\n",
    "(2) 通过在解码器端预测连续的标记，解码器可以比仅预测离散标记拥有更好的语言建模能力。<br>\n",
    "(3) 通过在解码器端进一步屏蔽在编码器端未被屏蔽掉的词， 以鼓励解码器从编码器端提取更多有用的信息来做预测，而不是依赖于前面预测出的单词，这样能促进编码器-解码器结构的联合训练。\n",
    "\n",
    "### 1.1 模型结构\n",
    "\n",
    "其模型基础结构可以使用任何Seq2Seq的结构，由于Transformer的优越性，故论文中使用Transformer模型作为基础结构，Transformer整体架构由编码器和解码器两个部分组成，不依赖任何RNN和CNN结构来生成输出，而是使用了Attention注意力机制，自动学习输入序列中每个单词和其他单词的关联，可以更好的处理长文本，且该模型可以高效的并行工作，训练速度较快。\n",
    "\n",
    "Transformer 的整体架构如下：\n",
    "\n",
    "<div align=center>\n",
    "<img src='https://i.imgur.com/ooO7ULP.png' width='400px'>\n",
    "</div>\n",
    "\n",
    "- 编码器和解码器分别由 $N=6$ 个相同的编码器/解码器层组成。\n",
    "- 在 Transformer 架构的左半部分，编码器的任务是将输入序列映射到一系列连续表示，然后将其馈送到解码器。\n",
    "- 架构右半部分的解码器接收编码器的输出以及前一个时间步的解码器输出，以生成输出序列。\n",
    "- 解码器的输出最终通过一个全连接层，然后是一个 softmax 层，以生成对输出序列下一个单词的预测。\n",
    "\n",
    "Transformer的运作过程举例如下图，这是一个由法文翻译为英文的 Transformer，从编码器端输入一句法文“je suis etudiant”，从解码器端则输出翻译出的英文“I am a student”。\n",
    "\n",
    "<div align=center>\n",
    "<img src='https://i.imgur.com/GQKvNAd.png' width='350'/>\n",
    "</div>\n",
    "\n",
    "1. 整句法文“je suis etudiant”做为编码器的输入，编码器一次处理整个句子，产生输出 $X$。\n",
    "2. 解码器以 $X$ 及空序列做为其输入，产生第一个输出 I。\n",
    "3. 解码器以 $X$ 及 I 做为其输入，产生第二个输出 am。\n",
    "4. 解码器以 $X$ 及 I am 做为其输入，产生第三个输出 a。\n",
    "5. 解码器以 $X$ 及 I am a 做为其输入，产生第四个输出 student。\n",
    "6. 解码器以 $X$ 及 I am a student 做为其输入，产生第五个输出 <end of sentence>。\n",
    "\n",
    "\n",
    "### 1.2目标函数\n",
    "\n",
    "给定一个未配对的源句子 $x ∈ \\mathcal X$ ，MASS通过被屏蔽的序列 $x^{\\setminus u:v}$ 作为输入来预测句子片段 $x^{u:v}$ 以预训练序列到序列模型。目标函数为一极大似然函数：\n",
    "\n",
    "$$\n",
    "L(\\theta; \\mathcal X) = \\frac{1}{|\\mathcal X|} \\sum_{x \\in \\mathcal X} \\log P(x^{u:v} | x^{\\setminus u:v}; \\theta) \\\\ = \\frac{1}{|\\mathcal X|} \\log \\Pi^{v}_{t=u} P(x^{u:v}_{t}|x^{u:v}_{\\textless t}, x^{\\setminus u:v};\\theta)\n",
    "$$\n",
    "\n",
    "注: $x^{u:v}$ 表示以句子位置 $u$ 为起点 $v$ 为终点的片段； $x^{\\setminus u:v}$ 为 $x^{u:v}$ 的修改版本，从 $u$ 到 $v$ 的片段被屏蔽， $0 < u < v < m$ 其中 $m$ 是句子 $x$ 长度。\n",
    "\n",
    "### 1.3 模型特点\n",
    "MASS 有一个重要的超参数 $k$，表示屏蔽的连续片段长度，通过调整 $k$ 的大小，MASS 能包含 BERT 中的掩码语言模型训练方法以及 GPT 中标准的语言模型预训练方法，使 MASS 成为一个通用的预训练框架。\n",
    "\n",
    "当 $k = 1$ 时，根据MASS的设定，编码器端仅屏蔽一个单词，解码器以源序列中未屏蔽的单词为条件预测这个单词，如图(a)所示。由于解码器的所有输入都被屏蔽了，因此解码器本身就像一个非线性分类器，类似于 BERT 中使用的 softmax 矩阵。在这种情况下，条件概率为 $P (x^u|x^{\\setminus u}; θ)$， $u$ 是掩码标记的位置，这正是 BERT3中使用的掩码语言模型的公式。\n",
    "\n",
    "当 $k = m$（ $m$ 为序列长度）时，根据MASS的设定，编码器会屏蔽所有的单词，解码器需要预测所有单词，如图(b)所示。由于编码器端所有词都被屏蔽了，解码器的注意力机制相当于没有获取到信息，在这种情况下条件概率为 $P(x^{1:m}|x^{\\setminus 1:m}; θ)$，等价于GPT中的标准语言模型。\n",
    "\n",
    "![](https://i.imgur.com/JdTGb9z.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 模型构建\n",
    "以下我们使用代码来建构Transformer模型。\n",
    "\n",
    "### 2.1 Embedding and Positional Encoding\n",
    "\n",
    "编码器与解码器在输入前的资料处理模块，包括 Input Embedding （Output Embedding) 和 Positional Encoding。\n",
    "\n",
    "<div align=center><img src='https://i.imgur.com/4OVf1jD.png' width='400px'/></div>\n",
    "\n",
    "在处理文本之前，需要先对其进行分词(tokenizer)，将文本切分成一个个子字串(token)，每个字串有相对完整的语意，便于后续将token转换为嵌入(Embedding)和位置编码(Positional Encoding)的向量表达。其中 Embedding 可以理解成一个词汇表(lookup table)，有了词汇表后就可以通过查表的方式得到每一个词的词向量。EmbeddingLookup类的主要功能就是创建词汇表，它的核心代码其实只有一行：\n",
    "```python\n",
    "self.embedding_table = Parameter(normal_weight([vocab_size, embedding_size], embedding_size))\n",
    "```\n",
    "词汇表（embedding_table）是一个Parameter类型，说明它里面的值是通过不断迭代学习得到的，效果是使相似的词的词向量有很高的相似度。它的shape为[vocal_size, embed_dim]，其中vocal_size代表所有词汇的总个数，是经过BPE编码得到的；embed_dim是指每一个词用多少维度的向量表示，本案例中将它的值设置为1024。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import mindspore.common.dtype as mstype\n",
    "from mindspore import nn\n",
    "from mindspore.ops import operations as P\n",
    "from mindspore.common.tensor import Tensor\n",
    "from mindspore.common.parameter import Parameter\n",
    "\n",
    "class EmbeddingLookup(nn.Cell):\n",
    "    def __init__(self,\n",
    "                 vocab_size,\n",
    "                 embed_dim):\n",
    "        super(EmbeddingLookup, self).__init__()\n",
    "        self.embedding_dim = embed_dim\n",
    "        self.vocab_size = vocab_size\n",
    "\n",
    "        init_weight = np.random.normal(0, embed_dim ** -0.5, size=[vocab_size, embed_dim]).astype(np.float32)\n",
    "        # 0 is Padding index, thus init it as 0.\n",
    "        init_weight[0, :] = 0\n",
    "        # 核心代码\n",
    "        self.embedding_table = Parameter(Tensor(init_weight))\n",
    "        \n",
    "        self.expand = P.ExpandDims()\n",
    "        self.gather = P.Gather()\n",
    "        self.one_hot = P.OneHot()\n",
    "        self.on_value = Tensor(1.0, mstype.float32)\n",
    "        self.off_value = Tensor(0.0, mstype.float32)\n",
    "        self.array_mul = P.MatMul()\n",
    "        self.reshape = P.Reshape()\n",
    "        self.get_shape = P.Shape()\n",
    "\n",
    "    def construct(self, input_ids):\n",
    "        # input_ids (Tensor): A batch of sentences with shape (N, T).\n",
    "        _shape = self.get_shape(input_ids)  # (N, T).\n",
    "        _batch_size = _shape[0]\n",
    "        _max_len = _shape[1]\n",
    "\n",
    "        flat_ids = self.reshape(input_ids, (_batch_size * _max_len,))\n",
    "        output_for_reshape = self.gather(self.embedding_table, flat_ids, 0)\n",
    "\n",
    "        output = self.reshape(output_for_reshape, (_batch_size, _max_len, self.embedding_dim))\n",
    "        # Returns: Tensor, word embeddings with shape (N, T, D)\n",
    "        return output, self.embedding_table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前面提到，Transformr不包含RNN和CNN结构，而是使用Attention 机制，其忽略了输入序列中每一个元素的位置及元素之间的距离，这样更容易找出序列中每一个元素的关连性，这是它的优点。然而在实际 NLP 应用中，句子内单词的顺序有着相当大的语法及语义上的含义，是不能忽略的。\n",
    "Transformer为了弥补这一缺点，添加了一个“位置编码 (Positional Encoding)”机制：输入序列在经过 embedding 处理后，每一个位置的输入值，都要加上该位置的位置编码，如此便可以在输入序列中加入位置信息。\n",
    "\n",
    "Position Encoding可以使用训练学习，也可以使用固定公式计算。Transformer采用的是不同频率的正弦和余弦函数：\n",
    "$$\n",
    "PE_{pos,2i} = sin(pos/10000^{2i/d_{\\mathrm{model}}}) \\\\\n",
    "PE_{pos,2i+1} = cos(pos/10000^{2i/d_{\\mathrm{model}}})\n",
    "$$\n",
    "其中，$pos$ 指的是这个 word 在这个句子中的位置，$i$ 指的是 embedding 维度($i ∈ {0, 1, 2, 3, …, d_{\\mathrm model}}$，它在偶数位置使用sin函数编码，在奇数位置使用cos函数编码)，$d_{\\mathrm model}$ 为模型的 embedding 大小。最后将其与Word Embedding相加后得到的结果就是编码器或解码器的输入。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def position_encoding(length, depth,\n",
    "                      min_timescale=1,\n",
    "                      max_timescale=1e4):\n",
    "    \"\"\"Create Tensor of sinusoids of different frequencies.\"\"\"\n",
    "    depth = depth // 2\n",
    "    positions = np.arange(length, dtype=np.float32)\n",
    "    log_timescale_increment = (np.log(max_timescale / min_timescale) / (depth - 1))\n",
    "    inv_timescales = min_timescale * np.exp(\n",
    "        np.arange(depth, dtype=np.float32) * -log_timescale_increment)\n",
    "    scaled_time = np.expand_dims(positions, 1) * np.expand_dims(inv_timescales, 0)\n",
    "    x = np.concatenate([np.sin(scaled_time), np.cos(scaled_time)], axis=1)\n",
    "    return x\n",
    "\n",
    "\n",
    "class PositionalEmbedding(nn.Cell):\n",
    "    \"\"\"Add positional info to word embeddings.\"\"\"\n",
    "    def __init__(self,\n",
    "                 embedding_size,\n",
    "                 max_position_embeddings=512):\n",
    "        super(PositionalEmbedding, self).__init__()\n",
    "        self.add = P.Add()\n",
    "        self.expand_dims = P.ExpandDims()\n",
    "        self.position_embedding_table = Tensor(\n",
    "            position_encoding(max_position_embeddings, embedding_size),\n",
    "            mstype.float32\n",
    "        )\n",
    "        self.gather = P.Gather()\n",
    "        self.get_shape = P.Shape()\n",
    "\n",
    "    def construct(self, word_embeddings):\n",
    "        input_shape = self.get_shape(word_embeddings)\n",
    "        input_len = input_shape[1]\n",
    "        # 核心代码\n",
    "        position_embeddings = self.position_embedding_table[0:input_len:1, ::]\n",
    "        position_embeddings = self.expand_dims(position_embeddings, 0)\n",
    "        output = self.add(word_embeddings, position_embeddings)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 2.2 注意力机制\n",
    "\n",
    "Transformer 所用的 Attention 机制，是使用基于「Self-Attention」的「Scaled Dot-Product Attention」，并在同一个子层中，使用 8 个平行的 「Scaled Dot-Product Attention」，成为「Multi-Head」Attention 结构。\n",
    "\n",
    "<div align=center>\n",
    "   <img src='https://i.imgur.com/99YUvcZ.png'/ width=500>\n",
    "</div>\n",
    "\n",
    "输入序列Q/K/V，对于编码器的第一层而言，它们是经过 Embedding 和位置编码的的输入序列，对于编码器的第二层以上的各层而言，它们是前一层的输出。\n",
    "\n",
    "在编码器中，计算单头注意力的方式如下：<br>\n",
    "输入 $[x_1, x_2, x_3, x_4...]$(其中 $x_i$是一个Word Embedding)，每一个 Scaled Dot-Product Attention 有一组属于它自己的权重矩阵 ($W_Q, W_K, W_V$)，输入 $x_i$ 和 $W_Q$ 作矩阵相乘，即得 $q_i$，以此类推，算出各个输入对应的 $q/k/v$，设置Query, key, Value三个矩阵。\n",
    "\n",
    "<div align=center>\n",
    "   <img src='https://i.imgur.com/AF6zqj5.png'/ width=500>\n",
    "</div>\n",
    "\n",
    "透过这三个矩阵计算注意力分数，执行步骤为：<br>\n",
    "第一步 (MatMul)：将 Q 和序列中所有的 K 作内积 (Dot-Product) ，得到一个内积序列。\n",
    "\n",
    "- 例如 $q_1$与 $k_1, k_2, k_3, k_4$ 做点积运算，则输出为 $[a_{1,1}, a_{1,2}, a_{1,3}, a_{1,4}]$， $a_{i, j}$ 表示第 $j$ 个Word Embedding对第 $i$ 个的重要性。\n",
    "\n",
    "第二步 (Scale)：将内积序列中的每个数除以「Ｋ长度的平方根」。<br>\n",
    "第三步 (SoftMax)：将序列做 SoftMax 运算。<br>\n",
    "第四步 (MatMul)：将输入 V 序列中的值，乘上其对应的 SoftMax 结果，得到一个 weighted V 序列。<br>\n",
    "第五步 (Sum)：将 weighted V 序列所有元素相加，得到输出的结果。\n",
    "\n",
    "#### 2.2.1 MultiHeadAttention\n",
    "\n",
    "多头自注意力的计算方式与单头自注意力相同，只是 $W_q, W_k, W_v$ 会重复head_num次。\n",
    "\n",
    "本案例中 $W_q = W_k = W_v =$ attn_embed_dim // num_attn_heads，attn_embed_dim设置为1024，num_attn_heads设置为8。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MultiHeadAttention(nn.Cell):\n",
    "    def __init__(self,\n",
    "                 src_dim,\n",
    "                 tgt_dim,\n",
    "                 attn_embed_dim,\n",
    "                 num_attn_heads=1,\n",
    "                 query_act=None,\n",
    "                 key_act=None,\n",
    "                 value_act=None,\n",
    "                 out_act=None,\n",
    "                 has_attention_mask=True,\n",
    "                 attention_dropout_prob=0.0,\n",
    "                 initializer_range=0.02,\n",
    "                 do_return_2d_tensor=True,\n",
    "                 compute_type=mstype.float32):\n",
    "        super(MultiHeadAttention, self).__init__()\n",
    "\n",
    "        self.attn_embed_dim = attn_embed_dim\n",
    "        self.num_attn_heads = num_attn_heads\n",
    "        self.size_per_head = attn_embed_dim // num_attn_heads\n",
    "        self.src_dim = src_dim\n",
    "        self.tgt_dim = tgt_dim\n",
    "        self.has_attention_mask = has_attention_mask\n",
    "        \n",
    "        self.scores_mul = Tensor([1.0 / math.sqrt(float(self.size_per_head))],\n",
    "                                 dtype=compute_type)\n",
    "        self.reshape = P.Reshape()\n",
    "        # 首先设置Q、K、V矩阵，它们的值是通过不断迭代学习到的, shape为[units, from_tensor_width]。代码中设置它的shape为[1024, 64]。\n",
    "        self.query_layer = nn.Dense(src_dim,\n",
    "                                    attn_embed_dim,\n",
    "                                    activation=query_act,\n",
    "                                    has_bias=True,\n",
    "                                    weight_init=TruncatedNormal(initializer_range)).to_float(compute_type)\n",
    "        self.key_layer = nn.Dense(tgt_dim,\n",
    "                                  attn_embed_dim,\n",
    "                                  activation=key_act,\n",
    "                                  has_bias=True,\n",
    "                                  weight_init=TruncatedNormal(initializer_range)).to_float(compute_type)\n",
    "        self.value_layer = nn.Dense(tgt_dim,\n",
    "                                    attn_embed_dim,\n",
    "                                    activation=value_act,\n",
    "                                    has_bias=True,\n",
    "                                    weight_init=TruncatedNormal(initializer_range)).to_float(compute_type)\n",
    "\n",
    "        self.out_layer = nn.Dense(attn_embed_dim,\n",
    "                                  attn_embed_dim,\n",
    "                                  activation=out_act,\n",
    "                                  has_bias=True,\n",
    "                                  weight_init=TruncatedNormal(initializer_range)).to_float(compute_type)\n",
    "\n",
    "        self.matmul_trans_b = P.BatchMatMul(transpose_b=True)\n",
    "        self.multiply = P.Mul()\n",
    "        self.transpose = P.Transpose()\n",
    "        self.multiply_data = Tensor([-10000.0], dtype=compute_type)\n",
    "        self.matmul = P.BatchMatMul()\n",
    "\n",
    "        self.softmax = nn.Softmax()\n",
    "        self.dropout = nn.Dropout(1.0 - attention_dropout_prob)\n",
    "\n",
    "        if self.has_attention_mask:\n",
    "            self.expand_dims = P.ExpandDims()\n",
    "            self.sub = P.Sub()\n",
    "            self.add = P.Add()\n",
    "            self.cast = P.Cast()\n",
    "            self.get_dtype = P.DType()\n",
    "\n",
    "        self.do_return_2d_tensor = do_return_2d_tensor\n",
    "        self.cast_compute_type = SaturateCast(dst_type=compute_type)\n",
    "        self.softmax_cast = P.Cast()\n",
    "        self.get_shape = P.Shape()\n",
    "        self.transpose_orders = (0, 2, 1, 3)\n",
    "\n",
    "    def construct(self, queries, keys, values, attention_mask):\n",
    "        q_shape = self.get_shape(queries)  # (N, T, D)\n",
    "        batch_size = q_shape[0]\n",
    "        src_max_len = q_shape[1]\n",
    "\n",
    "        k_shape = self.get_shape(keys)  # (N, T', D)\n",
    "        tgt_max_len = k_shape[1]\n",
    "\n",
    "        _src_4d_shape = (batch_size, src_max_len, self.num_attn_heads, self.size_per_head)\n",
    "        _tgt_4d_shape = (batch_size, tgt_max_len, self.num_attn_heads, self.size_per_head)\n",
    "\n",
    "        queries_2d = self.reshape(queries, (-1, self.src_dim))\n",
    "        keys_2d = self.reshape(keys, (-1, self.tgt_dim))\n",
    "        values_2d = self.reshape(values, (-1, self.tgt_dim))\n",
    "\n",
    "        query_out = self.query_layer(queries_2d)  # (N*T, D)*(D, D) -> (N*T, D)\n",
    "        key_out = self.key_layer(keys_2d)  # (N*T, D)*(D, D) -> (N*T, D)\n",
    "        value_out = self.value_layer(values_2d)  # (N*T, D)*(D, D) -> (N*T, D)\n",
    "\n",
    "        query_out = self.multiply(query_out, self.scores_mul)\n",
    "\n",
    "        query_layer = self.reshape(query_out, _src_4d_shape)\n",
    "        query_layer = self.transpose(query_layer, self.transpose_orders)  # (N, h, T, D')\n",
    "        key_layer = self.reshape(key_out, _tgt_4d_shape)\n",
    "        key_layer = self.transpose(key_layer, self.transpose_orders)  # (N, h, T', D')\n",
    "        value_layer = self.reshape(value_out, _tgt_4d_shape)\n",
    "        value_layer = self.transpose(value_layer, self.transpose_orders)  # (N, h, T', D')\n",
    "\n",
    "        attention_scores = self.matmul_trans_b(query_layer, key_layer)\n",
    "        # 编码器中不需要做Mask，而解码器中每一个子层皆要执行\n",
    "        if self.has_attention_mask:\n",
    "            attention_mask = self.expand_dims(attention_mask, 1)\n",
    "            multiply_out = self.sub(\n",
    "                self.cast(F.tuple_to_array((1.0,)), self.get_dtype(attention_scores)),\n",
    "                self.cast(attention_mask, self.get_dtype(attention_scores))\n",
    "            )  # make mask position into 1, unmask position into 0.\n",
    "            adder = self.multiply(multiply_out, self.multiply_data)\n",
    "            adder = self.softmax_cast(adder, mstype.float32)\n",
    "            attention_scores = self.softmax_cast(attention_scores, mstype.float32)\n",
    "            attention_scores = self.add(adder, attention_scores)\n",
    "\n",
    "        attention_scores = self.softmax_cast(attention_scores, mstype.float32)\n",
    "        attention_prob = self.softmax(attention_scores)\n",
    "        attention_prob = self.softmax_cast(attention_prob, self.get_dtype(key_layer))\n",
    "        attention_prob = self.dropout(attention_prob)\n",
    "        \n",
    "        # (N, h, T, T')*(N, h, T', D') -> (N, h, T, D')\n",
    "        context_layer = self.matmul(attention_prob, value_layer)\n",
    "        context_layer = self.transpose(context_layer, self.transpose_orders)  # (N, T, h, D')\n",
    "        context_layer = self.reshape(context_layer,\n",
    "                                     (batch_size * src_max_len, self.attn_embed_dim))  # (N*T, D)\n",
    "\n",
    "        context_layer = self.out_layer(context_layer)\n",
    "\n",
    "        if not self.do_return_2d_tensor:\n",
    "            context_layer = self.reshape(\n",
    "                context_layer, (batch_size, src_max_len, self.attn_embed_dim)\n",
    "            )  # (N, T, D)\n",
    "\n",
    "        return context_layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "source": [
    "#### 2.2.2 SelfAttention\n",
    "SelfAttention类中按顺序调用MultiheadAttention类、LayerNorm类(Layer Norm)、ResidualConnection类（Redual Layer)。<br>\n",
    "对应网络框架图中的 MultiheadAttention -> Add & Norm。\n",
    "\n",
    "<div align=center>\n",
    "    <img src='https://i.imgur.com/5KQFzmm.png' width='500px'>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class SelfAttention(nn.Cell):\n",
    "    def __init__(self,\n",
    "                 attn_embed_dim,\n",
    "                 num_attn_heads,\n",
    "                 attn_dropout_prob=0.1,\n",
    "                 initializer_range=0.02,\n",
    "                 dropout_prob=0.1,\n",
    "                 has_attention_mask=True,\n",
    "                 compute_type=mstype.float32):\n",
    "        super(SelfAttention, self).__init__()\n",
    "        self.multi_head_self_attention = MultiHeadAttention(\n",
    "            src_dim=attn_embed_dim,\n",
    "            tgt_dim=attn_embed_dim,\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            attention_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            has_attention_mask=has_attention_mask,\n",
    "            do_return_2d_tensor=False,\n",
    "            compute_type=compute_type)\n",
    "\n",
    "        self.layer_norm = LayerNorm(in_channels=attn_embed_dim)\n",
    "        self.residual = ResidualConnection(dropout_prob=dropout_prob)\n",
    "\n",
    "    def construct(self, queries, keys, values, attention_mask):\n",
    "        q = self.layer_norm(queries)  # (N, T, D)\n",
    "        attention_output = self.multi_head_self_attention(\n",
    "            q, keys, values, attention_mask\n",
    "        )  # (N, T, D)\n",
    "        q = self.residual(attention_output, queries)\n",
    "        return q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### 2.2.3 LayerNorm\n",
    "LayerNorm类实现了Layer Normalization方法，它可以有效的促进网络学习。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class LayerNorm(nn.Cell):\n",
    "\n",
    "    def __init__(self, in_channels=None, return_2d=False):\n",
    "        super(LayerNorm, self).__init__()\n",
    "        self.return_2d = return_2d\n",
    "        self.layer_norm = nn.LayerNorm((in_channels,))\n",
    "        self.cast = P.Cast()\n",
    "        self.get_dtype = P.DType()\n",
    "        self.reshape = P.Reshape()\n",
    "        self.get_shape = P.Shape()\n",
    "\n",
    "    def construct(self, input_tensor):\n",
    "        \"\"\"layer norm\"\"\"\n",
    "        shape = self.get_shape(input_tensor)\n",
    "        batch_size = shape[0]\n",
    "        max_len = shape[1]\n",
    "        embed_dim = shape[2]\n",
    "\n",
    "        output = self.reshape(input_tensor, (-1, embed_dim))\n",
    "        output = self.cast(output, mstype.float32)\n",
    "        output = self.layer_norm(output)\n",
    "        output = self.cast(output, self.get_dtype(input_tensor))\n",
    "        if not self.return_2d:\n",
    "            output = self.reshape(output, (batch_size, max_len, embed_dim))\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### 2.2.4 ResidualConnection\n",
    "ResidualConnection类实现一个残差结构，能有效解决梯度消失的问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class ResidualConnection(nn.Cell):\n",
    "\n",
    "    def __init__(self, dropout_prob=0.1):\n",
    "        super(ResidualConnection, self).__init__()\n",
    "        self.add = P.Add()\n",
    "        self.dropout = nn.Dropout(1.0 - dropout_prob)\n",
    "\n",
    "    def construct(self, hidden_tensor, residual):\n",
    "        output = self.dropout(hidden_tensor)\n",
    "        output = self.add(output, residual)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 FeedForward\n",
    "\n",
    "网络结构图中的Feed Forward -> Add & Norm 在FeedForwardNet类中实现。<br>\n",
    "先进行Layer Normalization 然后经过两个全连接层，最后加上残差结构输出。\n",
    "\n",
    "<div align=center>\n",
    "    <img src='https://i.imgur.com/5KQFzmm.png' width='550px'>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class FeedForwardNet(nn.Cell):\n",
    "    def __init__(self,\n",
    "                 in_channels,\n",
    "                 hidden_size,\n",
    "                 out_channels,\n",
    "                 hidden_act=\"relu\",\n",
    "                 initializer_range=0.02,\n",
    "                 hidden_dropout_prob=0.1,\n",
    "                 dropout=None,\n",
    "                 compute_type=mstype.float32):\n",
    "        super(FeedForwardNet, self).__init__()\n",
    "\n",
    "        self.fc1 = nn.Dense(in_channels,\n",
    "                            hidden_size,\n",
    "                            activation=hidden_act,\n",
    "                            weight_init=TruncatedNormal(initializer_range)).to_float(compute_type)\n",
    "        self.fc2 = nn.Dense(hidden_size,\n",
    "                            out_channels,\n",
    "                            weight_init=TruncatedNormal(initializer_range)).to_float(compute_type)\n",
    "\n",
    "        self.layer_norm = LayerNorm(in_channels=in_channels,\n",
    "                                    return_2d=True)\n",
    "        self.residual = ResidualConnection(\n",
    "            dropout_prob=hidden_dropout_prob if dropout is None else dropout\n",
    "        )\n",
    "        self.get_shape = P.Shape()\n",
    "        self.reshape = P.Reshape()\n",
    "        self.dropout = nn.Dropout(keep_prob=1.0 - hidden_dropout_prob)\n",
    "\n",
    "    def construct(self, input_tensor):\n",
    "        shape = self.get_shape(input_tensor)\n",
    "        batch_size = shape[0]\n",
    "        max_len = shape[1]\n",
    "        embed_dim = shape[2]\n",
    "        # 核心代码如下\n",
    "        output = self.layer_norm(input_tensor)\n",
    "        output = self.fc1(output)\n",
    "        output = self.dropout(output)\n",
    "        output = self.fc2(output)  # (-1, D)\n",
    "        output = self.residual(self.reshape(output, (batch_size, max_len, embed_dim)),\n",
    "                               input_tensor)  # (N, T, D)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "\n",
    "### 2.4 编码器与解码器\n",
    "\n",
    "<div align=center><img src='https://i.imgur.com/NNiUhIt.png' width='350px'/></div>\n",
    "\n",
    "可以看到编码器与解码器是由几个模块构成的，分别是Multi-Head Attention、Feed Forward、Masked Multi-Head Attention。\n",
    "\n",
    "#### 2.4.1 编码器\n",
    "\n",
    "编码器包含 6 个完全相同的神经网路层，每一个网路层包含了 2 个子层，第一个子层是 「multi-head self-attention」 层，第二个子层是 「positionwise fully connected feed-forward」 层。在每个层后还有一个正规化与残差运算。\n",
    "\n",
    "用数学式来看，每一个网路层可以表示为：<br>\n",
    "输入: $x$ <br>\n",
    "第一个子层输出: $y=LayerNorm(x+MHA(x))$ <br>\n",
    "第二个子层输出: $O=LayerNorm(y+PFF(y))$ <br>\n",
    "\n",
    "```\n",
    "编码器单层结构如下：\n",
    "    -> pre_LayerNorm\n",
    "    -> Multi-head Self-Attention\n",
    "    -> Dropout & Add\n",
    "    -> pre_LayerNorm\n",
    "    -> Fc1\n",
    "    -> Activation Function\n",
    "    -> Dropout\n",
    "    -> Fc2\n",
    "    -> Dropout & Add\n",
    "```\n",
    "\n",
    "EncoderCell类就是按顺序调用SelfAttention类与FeedForward类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "class EncoderCell(nn.Cell):\n",
    "    \"\"\"Single Encoder layer.\"\"\"\n",
    "    def __init__(self,\n",
    "                 attn_embed_dim=768,\n",
    "                 num_attn_heads=12,\n",
    "                 intermediate_size=3072,\n",
    "                 attention_dropout_prob=0.02,\n",
    "                 initializer_range=0.02,\n",
    "                 hidden_dropout_prob=0.1,\n",
    "                 hidden_act=\"relu\",\n",
    "                 compute_type=mstype.float32):\n",
    "        super(EncoderCell, self).__init__()\n",
    "        self.attention = SelfAttention(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            attn_dropout_prob=attention_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=hidden_dropout_prob,\n",
    "            compute_type=compute_type)\n",
    "        self.feed_forward_net = FeedForwardNet(\n",
    "            in_channels=attn_embed_dim,\n",
    "            hidden_size=intermediate_size,\n",
    "            out_channels=attn_embed_dim,\n",
    "            hidden_act=hidden_act,\n",
    "            initializer_range=initializer_range,\n",
    "            hidden_dropout_prob=hidden_dropout_prob,\n",
    "            dropout=hidden_dropout_prob,\n",
    "            compute_type=compute_type)\n",
    "\n",
    "    def construct(self, queries, attention_mask):\n",
    "        attention_output = self.attention(queries, queries, queries,\n",
    "                                          attention_mask)  # (N, T, D)\n",
    "        output = self.feed_forward_net(attention_output)  # (N, T, D)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TransformerEncoder类的功能是重复构造EncoderCell类，然后顺序连接起来，对应网络框架图中的Encoder部分的Nx结构，实际上是重复了6次。\n",
    "\n",
    "<div align=center><img src='https://i.imgur.com/DMbU83g.png' width='400px'/></div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TransformerEncoder(nn.Cell):\n",
    "    def __init__(self,\n",
    "                 attn_embed_dim,\n",
    "                 encoder_layers,\n",
    "                 num_attn_heads=12,\n",
    "                 intermediate_size=3072,\n",
    "                 attention_dropout_prob=0.1,\n",
    "                 initializer_range=0.02,\n",
    "                 hidden_dropout_prob=0.1,\n",
    "                 hidden_act=\"relu\",\n",
    "                 compute_type=mstype.float32):\n",
    "        super(TransformerEncoder, self).__init__()\n",
    "        self.num_layers = encoder_layers\n",
    "\n",
    "        layers = []\n",
    "        for _ in range(encoder_layers):\n",
    "            layer = EncoderCell(\n",
    "                attn_embed_dim=attn_embed_dim,\n",
    "                num_attn_heads=num_attn_heads,\n",
    "                intermediate_size=intermediate_size,\n",
    "                attention_dropout_prob=attention_dropout_prob,\n",
    "                initializer_range=initializer_range,\n",
    "                hidden_dropout_prob=hidden_dropout_prob,\n",
    "                hidden_act=hidden_act,\n",
    "                compute_type=compute_type\n",
    "            )\n",
    "            layers.append(layer)\n",
    "        # num_attention_heads的值为8，通过nn.CellList顺序连接起来。\n",
    "        self.layers = nn.CellList(layers)\n",
    "        self.layer_norm = LayerNorm(in_channels=attn_embed_dim)\n",
    "\n",
    "    def construct(self, input_tensor, attention_mask):\n",
    "        prev_output = input_tensor\n",
    "        for layer_module in self.layers:\n",
    "            prev_output = layer_module(prev_output,\n",
    "                                       attention_mask)  # (N, T, D)\n",
    "        prev_output = self.layer_norm(prev_output)  # (N, T, D)\n",
    "        return prev_output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.4.2 解码器\n",
    "\n",
    "解码器也同样包含了 6 个相同的神经网路层，每一个网路层含有 3 个子层，由下到上分别是：\n",
    "\n",
    ">Masked Multi-Head Self-Attention <br>\n",
    ">Multi-Head Attention <br>\n",
    ">Positionwise Fully Connected Feed-Forward\n",
    "\n",
    "每一个子层也实作残差连接及正规化处理。\n",
    "\n",
    "Transformer 解码器的架构和编码器雷同，但有两个主要的差异：\n",
    "- 解码器每一层的第一个子层必须执行 Masked 动作，使用 Masked Multi-Head Self-Attention 是为了确保在计算第 i 个输出时，仅用到 i 之前的输出值。\n",
    "- 解码器每一层都多了一个 Multi-Head Attention 子层，该层是以“第一个子层的输出”和“编码器的输出”执行 Attention 计算。\n",
    "\n",
    "CreateAttentionMaskFromInputMask类实现Mask Self Attention中的Mask功能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CreateAttentionMaskFromInputMask(nn.Cell):\n",
    "    \n",
    "    def __init__(self, config):\n",
    "        super(CreateAttentionMaskFromInputMask, self).__init__()\n",
    "        self.input_mask = None\n",
    "\n",
    "        self.cast = P.Cast()\n",
    "        self.shape = P.Shape()\n",
    "        self.reshape = P.Reshape()\n",
    "        self.batch_matmul = P.BatchMatMul()\n",
    "        self.multiply = P.Mul()\n",
    "        self.shape = P.Shape()\n",
    "        # mask future positions\n",
    "        ones = np.ones(shape=(config.batch_size, config.seq_length, config.seq_length))\n",
    "        self.lower_triangle_mask = Tensor(np.tril(ones), dtype=mstype.float32)\n",
    "\n",
    "    def construct(self, input_mask, mask_future=False):\n",
    "        input_shape = self.shape(input_mask)\n",
    "        # Add this for infer as the seq_length will increase.\n",
    "        shape_right = (input_shape[0], 1, input_shape[1])\n",
    "        shape_left = input_shape + (1,)\n",
    "\n",
    "        input_mask = self.cast(input_mask, mstype.float32)\n",
    "        mask_left = self.reshape(input_mask, shape_left)\n",
    "        mask_right = self.reshape(input_mask, shape_right)\n",
    "\n",
    "        attention_mask = self.batch_matmul(mask_left, mask_right)\n",
    "\n",
    "        if mask_future:\n",
    "            attention_mask = self.multiply(attention_mask, self.lower_triangle_mask)\n",
    "\n",
    "        return attention_mask"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "DecoderCell类中依序调用Self Attention类及FeedForward类。<br>\n",
    "其中第二个子层 Multi-Head Attention，它并不是一个 Self-Attention，它的输入 V 及 K 是来自编码器的输出值，而 Q 才是来自第一个子层。因此这一个子层在论文中，有时称它为 encoder-decoder attention (sub)layer，论文 $^{[1]}$ 中说明这种设计可以让每一个解码器子层的输入值去关注编码器的输出序列的每个位置。\n",
    "> In \"encoder-decoder attention\" layers, the queries come from the previous decoder layer, and the memory keys and values come from the output of the encoder. This allows every position in the decoder to attend over all positions in the input sequence. This mimics the typical encoder-decoder attention mechanisms in sequence-to-sequence models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DecoderCell(nn.Cell):\n",
    "\n",
    "    def __init__(self,\n",
    "                 attn_embed_dim=768,\n",
    "                 num_attn_heads=12,\n",
    "                 intermediate_size=3072,\n",
    "                 attn_dropout_prob=0.02,\n",
    "                 initializer_range=0.02,\n",
    "                 dropout_prob=0.1,\n",
    "                 hidden_act=\"relu\",\n",
    "                 compute_type=mstype.float32):\n",
    "        super(DecoderCell, self).__init__()\n",
    "        self.masked_attn = SelfAttention(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            attn_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=dropout_prob,\n",
    "            compute_type=compute_type)\n",
    "        self.enc_dec_attn = SelfAttention(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            attn_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=dropout_prob,\n",
    "            compute_type=compute_type)\n",
    "        self.feed_forward_net = FeedForwardNet(\n",
    "            in_channels=attn_embed_dim,\n",
    "            hidden_size=intermediate_size,\n",
    "            out_channels=attn_embed_dim,\n",
    "            hidden_act=hidden_act,\n",
    "            initializer_range=initializer_range,\n",
    "            hidden_dropout_prob=dropout_prob,\n",
    "            compute_type=compute_type)\n",
    "\n",
    "    def construct(self, queries, attention_mask, encoder_out, enc_attention_mask):\n",
    "        attention_output = self.masked_attn(\n",
    "            queries, queries, queries,\n",
    "            attention_mask\n",
    "        )\n",
    "        attention_output = self.enc_dec_attn(\n",
    "            attention_output,  # (N, T', D)\n",
    "            encoder_out, encoder_out,  # (N, T, D)\n",
    "            enc_attention_mask  # (N, T, T)\n",
    "        )\n",
    "        output = self.feed_forward_net(attention_output)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TransformerDecoder类的功能是重复构造DecoderCell类，然后顺序连接起来，对应网络框架图中的Decoder部分的Nx结构，实际上是重复了6次。\n",
    "\n",
    "\n",
    "<div align=center><img src='https://i.imgur.com/bdu2GMc.png' width='400px'/></div>\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TransformerDecoder(nn.Cell):\n",
    "    def __init__(self,\n",
    "                 attn_embed_dim,\n",
    "                 decoder_layers,\n",
    "                 num_attn_heads,\n",
    "                 intermediate_size,\n",
    "                 attn_dropout_prob=0.1,\n",
    "                 initializer_range=0.02,\n",
    "                 dropout_prob=0.1,\n",
    "                 hidden_act=\"relu\",\n",
    "                 compute_type=mstype.float32):\n",
    "        super(TransformerDecoder, self).__init__()\n",
    "        self.num_layers = decoder_layers\n",
    "        self.attn_embed_dim = attn_embed_dim\n",
    "\n",
    "        self.layer0 = DecoderCell(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            intermediate_size=intermediate_size,\n",
    "            attn_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=dropout_prob,\n",
    "            hidden_act=hidden_act,\n",
    "            compute_type=compute_type\n",
    "        )\n",
    "        self.layer1 = DecoderCell(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            intermediate_size=intermediate_size,\n",
    "            attn_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=dropout_prob,\n",
    "            hidden_act=hidden_act,\n",
    "            compute_type=compute_type\n",
    "        )\n",
    "        self.layer2 = DecoderCell(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            intermediate_size=intermediate_size,\n",
    "            attn_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=dropout_prob,\n",
    "            hidden_act=hidden_act,\n",
    "            compute_type=compute_type\n",
    "        )\n",
    "        self.layer3 = DecoderCell(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            intermediate_size=intermediate_size,\n",
    "            attn_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=dropout_prob,\n",
    "            hidden_act=hidden_act,\n",
    "            compute_type=compute_type\n",
    "        )\n",
    "        self.layer4 = DecoderCell(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            intermediate_size=intermediate_size,\n",
    "            attn_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=dropout_prob,\n",
    "            hidden_act=hidden_act,\n",
    "            compute_type=compute_type\n",
    "        )\n",
    "        self.layer5 = DecoderCell(\n",
    "            attn_embed_dim=attn_embed_dim,\n",
    "            num_attn_heads=num_attn_heads,\n",
    "            intermediate_size=intermediate_size,\n",
    "            attn_dropout_prob=attn_dropout_prob,\n",
    "            initializer_range=initializer_range,\n",
    "            dropout_prob=dropout_prob,\n",
    "            hidden_act=hidden_act,\n",
    "            compute_type=compute_type\n",
    "        )\n",
    "\n",
    "        self.layer_preprocess = LayerNorm(in_channels=attn_embed_dim,\n",
    "                                          return_2d=False)\n",
    "\n",
    "    def construct(self, input_tensor, attention_mask, encoder_out, enc_attention_mask):\n",
    "        prev_output = input_tensor\n",
    "        prev_output = self.layer0(prev_output, attention_mask, encoder_out, enc_attention_mask)\n",
    "        prev_output = self.layer1(prev_output, attention_mask, encoder_out, enc_attention_mask)\n",
    "        prev_output = self.layer2(prev_output, attention_mask, encoder_out, enc_attention_mask)\n",
    "        prev_output = self.layer3(prev_output, attention_mask, encoder_out, enc_attention_mask)\n",
    "        prev_output = self.layer4(prev_output, attention_mask, encoder_out, enc_attention_mask)\n",
    "        prev_output = self.layer5(prev_output, attention_mask, encoder_out, enc_attention_mask)\n",
    "\n",
    "        # Add layer norm, and full connection layer.\n",
    "        prev_output = self.layer_preprocess(prev_output)\n",
    "        return prev_output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 SoftMax\n",
    "解码器的输出是一个浮点数向量，经过一个解码器中最后的Linear 层，输出为一个 logits 向量，此一向量的长度即是字汇数量，每一个位置代表一个字。假设我们的输出为一 10000 字的英文字汇集合，那么 logit 向量的长度即为 10000。\n",
    "\n",
    "再经过 Softmax 处理，则输出为一机率分布向量，代表每一个对应文字的机率。通常最高机率的那一个字，即被选为此次的输出单字。\n",
    "\n",
    "PredLogProbs类实现SoftMax功能，对应网络框架图中Decoder部分最后输出之前的SoftMax部分。\n",
    "\n",
    "<div align=center>\n",
    "<img src='https://i.imgur.com/i0Vbk7r.png' width=150/>\n",
    "</div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PredLogProbs(nn.Cell):\n",
    "    def __init__(self,\n",
    "                 batch_size,\n",
    "                 seq_length,\n",
    "                 width,\n",
    "                 compute_type=mstype.float32,\n",
    "                 dtype=mstype.float32):\n",
    "        super(PredLogProbs, self).__init__()\n",
    "        self.batch_size = batch_size\n",
    "        self.seq_length = seq_length\n",
    "        self.width = width\n",
    "        self.compute_type = compute_type\n",
    "        self.dtype = dtype\n",
    "\n",
    "        self.reshape = P.Reshape()\n",
    "        self.matmul = P.MatMul(transpose_b=True)\n",
    "        self.log_softmax = nn.LogSoftmax(axis=-1)\n",
    "        self.shape_flat_sequence_tensor = (self.batch_size * self.seq_length, self.width)\n",
    "        self.cast = P.Cast()\n",
    "\n",
    "    def construct(self, input_tensor, output_weights):\n",
    "        input_tensor = self.reshape(input_tensor, self.shape_flat_sequence_tensor)\n",
    "        input_tensor = self.cast(input_tensor, self.compute_type)\n",
    "        output_weights = self.cast(output_weights, self.compute_type)\n",
    "\n",
    "        logits = self.matmul(input_tensor, output_weights)\n",
    "        logits = self.cast(logits, self.dtype)\n",
    "\n",
    "        log_probs = self.log_softmax(logits)\n",
    "        return log_probs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6 Transformer\n",
    "Transformer类连接编码器和解码器，并分别在其输入前加上 Embedding & Positional Encoding，并在解码器的输出处加上 Linear 层及 Softmax 层，实现从输入到输出的端到端流程。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Transformer(nn.Cell):\n",
    "\n",
    "    def __init__(self):\n",
    "        super(Transformer, self).__init__()\n",
    "\n",
    "        self.batch_size = config.batch_size\n",
    "        self.max_positions = config.seq_length\n",
    "        self.attn_embed_dim = config.hidden_size\n",
    "        self.num_layers = config.num_hidden_layers\n",
    "        self.word_embed_dim = config.hidden_size\n",
    "\n",
    "        self.last_idx = self.num_layers - 1\n",
    "\n",
    "        self.embedding_lookup = EmbeddingLookup(\n",
    "            vocab_size=config.vocab_size,\n",
    "            embed_dim=self.word_embed_dim,\n",
    "            use_one_hot_embeddings=False)\n",
    "\n",
    "        self.positional_embedding = PositionalEmbedding(\n",
    "            embedding_size=self.word_embed_dim,\n",
    "            max_position_embeddings=config.max_position_embeddings)\n",
    "\n",
    "        self.encoder = TransformerEncoder(\n",
    "            attn_embed_dim=self.attn_embed_dim,\n",
    "            encoder_layers=self.num_layers,\n",
    "            num_attn_heads=config.num_attention_heads,\n",
    "            intermediate_size=config.intermediate_size,\n",
    "            attention_dropout_prob=config.attention_dropout_prob,\n",
    "            initializer_range=config.initializer_range,\n",
    "            hidden_dropout_prob=config.hidden_dropout_prob,\n",
    "            hidden_act=config.hidden_act,\n",
    "            compute_type=config.compute_type)\n",
    "\n",
    "        self.decoder = TransformerDecoder(\n",
    "            attn_embed_dim=self.attn_embed_dim,\n",
    "            decoder_layers=self.num_layers,\n",
    "            num_attn_heads=config.num_attention_heads,\n",
    "            intermediate_size=config.intermediate_size,\n",
    "            attn_dropout_prob=config.attention_dropout_prob,\n",
    "            initializer_range=config.initializer_range,\n",
    "            dropout_prob=config.hidden_dropout_prob,\n",
    "            hidden_act=config.hidden_act,\n",
    "            compute_type=config.compute_type)\n",
    "\n",
    "        self.cast = P.Cast()\n",
    "        self.dtype = config.dtype\n",
    "        self.cast_compute_type = SaturateCast(dst_type=config.compute_type)\n",
    "        self.slice = P.StridedSlice()\n",
    "        self.dropout = nn.Dropout(keep_prob=1.0 - config.hidden_dropout_prob)\n",
    "\n",
    "        self._create_attention_mask_from_input_mask = CreateAttentionMaskFromInputMask(config)\n",
    "\n",
    "        self.scale = Tensor([math.sqrt(float(self.word_embed_dim))],\n",
    "                            dtype=mstype.float32)\n",
    "        self.multiply = P.Mul()\n",
    "\n",
    "    def construct(self, source_ids, source_mask, target_ids, target_mask):\n",
    "        # Process source sentences.\n",
    "        src_embeddings, embedding_tables = self.embedding_lookup(source_ids)\n",
    "        src_embeddings = self.multiply(src_embeddings, self.scale)\n",
    "        if self.use_positional_embedding:\n",
    "            src_embeddings = self.positional_embedding(src_embeddings)\n",
    "        src_embeddings = self.dropout(src_embeddings)\n",
    "\n",
    "        # Attention mask with shape (N, T, T).\n",
    "        enc_attention_mask = self._create_attention_mask_from_input_mask(source_mask)\n",
    "        # Transformer encoder.\n",
    "        encoder_output = self.encoder(\n",
    "            self.cast_compute_type(src_embeddings),  # (N, T, D).\n",
    "            self.cast_compute_type(enc_attention_mask)  # (N, T, T).\n",
    "        )\n",
    "\n",
    "        # Process target sentences.\n",
    "        tgt_embeddings, _ = self.embedding_lookup(target_ids)\n",
    "        tgt_embeddings = self.multiply(tgt_embeddings, self.scale)\n",
    "        if self.use_positional_embedding:\n",
    "            tgt_embeddings = self.positional_embedding(tgt_embeddings)\n",
    "        tgt_embeddings = self.dropout(tgt_embeddings)\n",
    "\n",
    "        # Attention mask with shape (N, T', T').\n",
    "        tgt_attention_mask = self._create_attention_mask_from_input_mask(\n",
    "            target_mask, True\n",
    "        )\n",
    "        # Transformer decoder.\n",
    "        decoder_output = self.decoder(\n",
    "            self.cast_compute_type(tgt_embeddings),  # (N, T', D)\n",
    "            self.cast_compute_type(tgt_attention_mask),  # (N, T', T')\n",
    "            encoder_output,  # (N, T, D)\n",
    "            enc_attention_mask  # (N, T, T)\n",
    "        )\n",
    "\n",
    "        return encoder_output, decoder_output, embedding_tables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注：本节中的模型图，除了论文 $^{[1]}$ 中的图片，其他皆源自 Jay Alammar 的[《The Illustrated Transformer》](https://jalammar.github.io/illustrated-transformer/)<br>\n",
    "[1] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin. 2017. [Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf). arXiv:1706.03762."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 案例实现\n",
    "\n",
    "### 3.1 环境建置\n",
    "本案例的开发环境为Modelarts上的notebook。<br>\n",
    "版本：mindspore 1.8.1、python 3.7<br>\n",
    "Ascend 环境：1\\*Ascend 910 | CPU: 24vCPUs 96GB<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MindSpore version:  1.8.1\n",
      "The result of multiplication calculation is correct, MindSpore has been installed successfully!\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "import mindspore\n",
    "print(mindspore.run_check())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "进入开发环境后还需安装一些所需套件，执行以下命令即可："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Requirement already satisfied: nltk in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (3.7)\n",
      "Requirement already satisfied: jieba in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (0.42.1)\n",
      "Requirement already satisfied: numpy in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (1.17.5)\n",
      "Requirement already satisfied: subword-nmt in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from -r requirements.txt (line 4)) (0.3.8)\n",
      "Requirement already satisfied: files2rouge in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from -r requirements.txt (line 5)) (1.1)\n",
      "Requirement already satisfied: pyyaml in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from -r requirements.txt (line 7)) (5.3.1)\n",
      "Requirement already satisfied: rouge in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from -r requirements.txt (line 8)) (1.0.1)\n",
      "Requirement already satisfied: joblib in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from nltk->-r requirements.txt (line 1)) (1.1.0)\n",
      "Requirement already satisfied: tqdm in /home/ma-user/modelarts-dev/modelarts-sdk (from nltk->-r requirements.txt (line 1)) (4.62.3)\n",
      "Requirement already satisfied: regex>=2021.8.3 in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from nltk->-r requirements.txt (line 1)) (2022.10.31)\n",
      "Requirement already satisfied: click in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from nltk->-r requirements.txt (line 1)) (8.1.3)\n",
      "Requirement already satisfied: mock in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from subword-nmt->-r requirements.txt (line 4)) (4.0.3)\n",
      "Requirement already satisfied: six in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from rouge->-r requirements.txt (line 8)) (1.16.0)\n",
      "Requirement already satisfied: importlib-metadata in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from click->nltk->-r requirements.txt (line 1)) (4.12.0)\n",
      "Requirement already satisfied: typing-extensions>=3.6.4 in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from importlib-metadata->click->nltk->-r requirements.txt (line 1)) (3.10.0.2)\n",
      "Requirement already satisfied: zipp>=0.5 in /home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages (from importlib-metadata->click->nltk->-r requirements.txt (line 1)) (3.8.1)\n"
     ]
    }
   ],
   "source": [
    "!pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import nltk\n",
    "# nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "若是nltk的punkt套件无法下载的话请至[该连接](https://github.com/nltk/nltk_data/tree/gh-pages/packages/tokenizers)下载punkt.zip，解压后将punkt资料夹放入/root/nltk_data/tokenizers/路径下。<br>\n",
    "ModelArts环境下放置在/home/ma-user/nltk_data/tokenizers/路径下，步骤为：将下载好的punkt.zip上传至work目录中，然後在终端输入如下命令。\n",
    "```\n",
    "[ma-user work]$ cd ..\n",
    "[ma-user ~]$ mkdir nltk_data\n",
    "[ma-user ~]$ mkdir nltk_data/tokenizers\n",
    "[ma-user ~]$ cp work/punkt.zip nltk_data/tokenizers\n",
    "[ma-user ~]$ cd nltk_data/tokenizers\n",
    "[ma-user tokenizers]$ unzip punkt.zip\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1.1 代码结构\n",
    "\n",
    "MASS脚本及代码结构如下：\n",
    "\n",
    "```text\n",
    "├── mass\n",
    "  ├── config\n",
    "  │   ├──config.py                           // 参数配置\n",
    "  ├── src\n",
    "      │   ├──model_utils\n",
    "      │   ├──config.py                       // 参数配置\n",
    "      │   ├──device_adapter.py               // 设备配置\n",
    "      │   ├──local_adapter.py                // 本地设备配置\n",
    "      │   ├──moxing_adapter.py               // modelarts设备配置\n",
    "  ├──src\n",
    "  │   ├──dataset\n",
    "  │      ├──bi_data_loader.py                // 数据集加载器，用于微调或推理\n",
    "  │      ├──mono_data_loader.py              // 预训练数据集加载器\n",
    "  │   ├──language_model\n",
    "  │      ├──noise_channel_language_model.p   // 数据集生成噪声通道语言模型\n",
    "  │      ├──mass_language_model.py           // 基于MASS论文的MASS语言模型\n",
    "  │      ├──loose_masked_language_model.py   // 基于MASS发布代码的MASS语言模型\n",
    "  │      ├──masked_language_model.py         // 基于MASS论文的MASS语言模型\n",
    "  │   ├──transformer\n",
    "  │      ├──create_attn_mask.py              // 生成屏蔽矩阵，除去填充部分\n",
    "  │      ├──transformer.py                   // Transformer模型架构\n",
    "  │      ├──encoder.py                       // Transformer编码器组件\n",
    "  │      ├──decoder.py                       // Transformer解码器组件\n",
    "  │      ├──self_attention.py                // 自注意块组件\n",
    "  │      ├──multi_head_attention.py          // 多头自注意组件\n",
    "  │      ├──embedding.py                     // 嵌入组件\n",
    "  │      ├──positional_embedding.py          // 位置嵌入组件\n",
    "  │      ├──feed_forward_network.py          // 前馈网络\n",
    "  │      ├──residual_conn.py                 // 残留块\n",
    "  │      ├──beam_search.py                   // 推理所用的波束搜索解码器\n",
    "  │      ├──transformer_for_infer.py         // 使用Transformer进行推理\n",
    "  │      ├──transformer_for_train.py         // 使用Transformer进行训练\n",
    "  │   ├──utils\n",
    "  │      ├──byte_pair_encoding.py            // 使用subword-nmt应用字节对编码（BPE）\n",
    "  │      ├──dictionary.py                    // 字典\n",
    "  │      ├──loss_moniter.py                  // 训练步骤中损失监控回调\n",
    "  │      ├──lr_scheduler.py                  // 学习速率调度器\n",
    "  │      ├──ppl_score.py                     // 基于N-gram的困惑度评分\n",
    "  │      ├──rouge_score.py                   // 计算ROUGE得分\n",
    "  │      ├──load_weights.py                  // 从检查点或者NPZ文件加载权重\n",
    "  │      ├──initializer.py                   // 参数初始化器\n",
    "  ├── vocab\n",
    "  │   ├──all.bpe.codes                       // 字节对编码表\n",
    "  │   ├──all_en.dict.bin                     // 已学习到的词汇表\n",
    "  ├── scripts\n",
    "  │   ├──run_ascend.sh                       // Ascend处理器上训练&评估模型脚本\n",
    "  │   ├──run_gpu.sh                          // GPU处理器上训练&评估模型脚本\n",
    "  │   ├──learn_subword.sh                    // 学习字节对编码\n",
    "  │   ├──stop_training.sh                    // 停止训练\n",
    "  ├── requirements.txt                       // 第三方包需求\n",
    "  ├── train.py                               // 训练API入口\n",
    "  ├── eval.py                                // 推理API入口\n",
    "  ├── default_config.yaml                    // 参数配置\n",
    "  ├── tokenize_corpus.py                     // 语料标记化\n",
    "  ├── apply_bpe_encoding.py                  // 应用BPE进行编码\n",
    "  ├── weights_average.py                     // 将各模型检查点平均转换到NPZ格式\n",
    "  ├── news_crawl.py                          // 创建预训练所用的News Crawl数据集\n",
    "  ├── gigaword.py                            // 创建Gigaword语料库\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 准备数据集\n",
    "案例实现中预训练所使用的数据即News Crawl的英语单语数据数据集，因為考量到訓練模型所需時間，只使用2012年的數據(News Crawl: articles from 2012)，并且只使用该文件十六分之一的数据量，下载好的数据集为一纯文字文件，接下来需要对该数据进行预处理，预处理包括对数据进行分词、利用subword-nmt工具生成字节对编码(BPE)、对分词后的语料应用该BPE编码進行切分并构建词彙表幾個步驟。\n",
    "\n",
    "而微调模型用于文本摘要任务所使用的数据集为Gigaword，该数据集已经有分割为训练、测试、验证集，有原文本(src)和目标摘要(tgt)两个文件，本案例只会使用训练集与测试集，也需要對這兩個文件進行同樣的預處理。\n",
    "\n",
    "数据集文件路径结构如下：\n",
    "\n",
    "```text\n",
    ".dataset/\n",
    "└── news_crawl\n",
    "    └── news2012_short.txt(106.7MB)\n",
    "└── ggw_data\n",
    "    ├── test.src.txt(335KB)\n",
    "    ├── test.tgt.txt(104KB)\n",
    "    ├── train_short.src.txt(83MB)\n",
    "    └── train_short.tgt.txt(23MB)\n",
    "```\n",
    "注：_short为减少数据量后的文件，可以在obs桶内下载，路径为obs://mass-datasets/dataset_short/，原数据集是obs://mass-datasets/news_crawl.zip、obs://mass-datasets/ggw_data.zip。\n",
    "\n",
    "下载连接：<br>\n",
    "News Crawl数据集（WMT，2019年）：https://www.statmt.org/wmt16/translation-task.html<br>\n",
    "Gigaword语料库（Graff等人，2003年）： https://drive.google.com/open?id=0B6N7tANPyVeBNmlSX19Ld2xDU1E\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1 对原始数据进行分词\n",
    "\n",
    "文本是由段落（Paragraph）构成的，段落是由句子（Sentence）构成的，句子是由单词构成的。分词是文本分析的第一步，它把文本段落分解为较小的实体（如单词或句子），每一个实体叫做一个Token。NLTK套件能够实现句子切分和单词切分两种功能。\n",
    "\n",
    "因为原始数据是一行一行句子的格式，所以只要调用nltk套件中的word_tokenize()函数文本进行分词，此函数的作用是基于空格/标点等对文本进行分词，返回分词后的列表，最后只保留长度大于175的单词列表，并将这些单词列表写入目标文件中。\n",
    "\n",
    "```\n",
    "sen = \"Dave Aneckstein, Simmons Research, an Experian Company\"\n",
    "tokens = word_tokenize(sen)\n",
    "tokens = ['Dave', 'Aneckstein', ',', 'Simmons', 'Research', ',', 'an', 'Experian', 'Company']\n",
    "```\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Processing ./dataset/news_crawl/news2012_short.txt.\n",
      " | Wrote to ./tokenized_corpus/news_crawl/news2012_short_tokenized.txt.\n",
      " | Processing ./dataset/ggw_data/train_short.src.txt.\n",
      " | Wrote to ./tokenized_corpus/ggw_data/train_short.src_tokenized.txt.\n",
      " | Processing ./dataset/ggw_data/test.src.txt.\n",
      " | Wrote to ./tokenized_corpus/ggw_data/test.src_tokenized.txt.\n",
      " | Processing ./dataset/ggw_data/train_short.tgt.txt.\n",
      " | Wrote to ./tokenized_corpus/ggw_data/train_short.tgt_tokenized.txt.\n",
      " | Processing ./dataset/ggw_data/test.tgt.txt.\n",
      " | Wrote to ./tokenized_corpus/ggw_data/test.tgt_tokenized.txt.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from nltk.tokenize import word_tokenize\n",
    "\n",
    "data_list = ['news_crawl','ggw_data']\n",
    "src_folder = \"./dataset/\"\n",
    "out_folder = \"./tokenized_corpus/\"\n",
    "\n",
    "if not os.path.isdir(out_folder):\n",
    "    os.mkdir(out_folder)\n",
    "if not os.path.isdir(out_folder+'news_crawl/'):\n",
    "    os.mkdir(out_folder+'news_crawl/')\n",
    "if not os.path.isdir(out_folder+'ggw_data/'):\n",
    "    os.mkdir(out_folder+'ggw_data/')\n",
    "\n",
    "def create_tokenized_sentences(file_path, tokenized_file):\n",
    "    tokenized_sen = []\n",
    "    print(f\" | Processing {file_path}.\")\n",
    "    with open(file_path, \"r\") as file:\n",
    "        for sen in file:\n",
    "            tokens = word_tokenize(sen)\n",
    "            tokens = [t for t in tokens if t != \" \"]\n",
    "            if len(tokens) > 175:\n",
    "                continue\n",
    "            tokenized_sen.append(\" \".join(tokens) + \"\\n\")\n",
    "\n",
    "    with open(tokenized_file, \"w\") as file:\n",
    "        file.writelines(tokenized_sen)\n",
    "    print(f\" | Wrote to {tokenized_file}.\")\n",
    "\n",
    "for item in data_list:\n",
    "    folder_path = os.path.join(src_folder, item)\n",
    "    output_path = os.path.join(out_folder, item)\n",
    "    for file in os.listdir(folder_path):\n",
    "        if not file.endswith(\".txt\"):\n",
    "            continue\n",
    "        file_path = os.path.join(folder_path, file)\n",
    "        tokenized_file = os.path.join(output_path, file.replace(\".txt\", \"_tokenized.txt\"))\n",
    "        create_tokenized_sentences(file_path, tokenized_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.2 生成字节对编码\n",
    "虽然我们已经将数据进行了简单的分词，但是实际上这样的分词是不够细緻的，若要使用一个包含所有单词的字典，将需要很大的空间和计算量，而且过大的token列表也会影响模型的预测准确度。\n",
    "\n",
    "字节对编码（BPE, Byte Pair Encoder）是一种简单的数据压缩算法，它将字符串中出现频率最高的相邻字符替换成一个不存在的新字符，反复进行该操作直到满足某些预设条件为止（字符表大小、迭代次数）。后续使用时还需要一个词表来重建原始数据。\n",
    "\n",
    "BPE的处理过程可以理解为一个单词的再拆分过程。如\"loved\",\"loving\",\"loves\"这三个单词，其本身的语义都是”爱”的意思。BPE通过训练，能够把上面的3个单词拆分成”lov”,”ed”,”ing”,”es”几部分，这样可以把词的本身的意思和时态分开，有效的减少了词表大小，且罕见的词会被分解为两个或多个subword tokens，能比较好的处理OOV(out of vocabulary)问题。\n",
    "\n",
    "我们可以利用subword-nmt工具学习文本的字节对编码(learn-bpe)：\n",
    "\n",
    "先设定选择最高频字节对的次数num_operations，本案例设定为46000。此过程会统计每一个相邻字节对的出现频率，并保存为codes_file。\n",
    "```\n",
    "subword-nmt learn-bpe -i {input_src_file} -s {num_operations} -o {codes_file}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|##########| 46000/46000 [08:08<00:00, 94.18it/s] \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "if not os.path.isdir('./vocab/'):\n",
    "    os.mkdir('./vocab/')\n",
    "\n",
    "import subprocess\n",
    "commands = \"cat ./dataset/ggw_data/*.txt ./dataset/news_crawl/*.txt | subword-nmt learn-bpe -s 46000 -o ./vocab/all.bpe.codes\"\n",
    "\n",
    "subprocess.call(commands,shell=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.3 应用字节对编码并构建词彙表\n",
    "使用上一节获取的BPE编码对分词后的语料进行字节对编码处理(apply-bpe)，生成subword词表。此过程会将input_tokenized_file中的单词拆分为字符序列并在末尾添加后缀“<\\/w>”，而后按照code_file将出现频率最高的字节对合并成新的subword，重複合併直到达到设定的subword词表大小或下一个最高频的字节对出现频率为1，最后将结果保存为out_file，文件名为input_tokenized_file_bpe.txt。\n",
    "```\n",
    "subword-nmt apply-bpe -c {code_file} -i {input_tokenized_file} -o {out_file}\n",
    "可以通过 --vocabulary-threshold {threshold} 選項过滤词频低于阈值的单词来缩小词汇量。\n",
    "```\n",
    "构建词彙表(get-vocab)的具体作法为：得到subword词表后，对该词表按照子词长度由大到小排序。编码时，对于每个单词，遍历词表寻找是否有token是当前单词的子字符串，如果有，则该token为表示当前单词的tokens之一。从最长的token迭代到最短的token，尝试将每个单词中的子字符串替换为已存在的token。若每个单词都替换完后仍然有子字符串没被替换，则将剩余的子词替换为特殊token，如\\<unk>。\n",
    "```\n",
    "subword-nmt get-vocab -i {output_path} -o {dict_path}\n",
    "# output_path為apply-bpe的輸出路徑\n",
    "# dict_path為輸出的.dict檔案路徑\n",
    "```\n",
    "\n",
    "根据News Crawl 2012数据集的14869673个句子，学习到的词汇量为39284个单词。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Applying BPE encoding.\n",
      " | Fetching vocabulary from single file.\n",
      " | Vocabulary Size: 36133\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import subprocess\n",
    "from src.utils import Dictionary\n",
    "\n",
    "source_folder = os.path.abspath(\"./tokenized_corpus/news_crawl/\")\n",
    "output_folder = os.path.abspath(\"./tokenized_corpus/news_crawl/bpe/\")\n",
    "codes = os.path.abspath(\"./vocab/all.bpe.codes\")\n",
    "vocab_path = \"./vocab/all_en.dict.bin\"\n",
    "\n",
    "if not os.path.isdir(output_folder):\n",
    "    os.mkdir(output_folder)\n",
    "\n",
    "ENCODER = \"subword-nmt apply-bpe -c\"\n",
    "LEARN_DICT = \"subword-nmt get-vocab -i\"\n",
    "def bpe_encode(codes_path, src_path, output_path, dict_path):\n",
    "    # Encoding.\n",
    "    print(\" | Applying BPE encoding.\")\n",
    "    commands = ENCODER.split() + [codes_path] + [\"-i\"] + [src_path] + [\"-o\"] + [output_path]\n",
    "    subprocess.call(commands)\n",
    "    print(\" | Fetching vocabulary from single file.\")\n",
    "    # Learn vocab.\n",
    "    commands = LEARN_DICT.split() + [output_path] + [\"-o\"] + [dict_path]\n",
    "    subprocess.call(commands)\n",
    "\n",
    "available_dict = []\n",
    "for file in os.listdir(source_folder):\n",
    "    if file.endswith(\".txt\"):\n",
    "        output_path = os.path.join(output_folder, file.replace(\".txt\", \"_bpe.txt\"))\n",
    "        dict_path = os.path.join(output_folder, file.replace(\".txt\", \".dict\"))\n",
    "        available_dict.append(dict_path)\n",
    "        bpe_encode(codes, os.path.join(source_folder, file), output_path, dict_path)\n",
    "        \n",
    "# 加载bpe_encode處理過的文本词汇表，行格式为[word, freq]。\n",
    "vocab = Dictionary.load_from_text(available_dict)\n",
    "vocab.persistence(vocab_path) #将词汇表对象保存为二进制文件。\n",
    "print(f\" | Vocabulary Size: {len(vocab)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 生成数据集\n",
    "\n",
    "生成的数据集格式是mindrecord。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Processing corpus ./dataset/news_crawl/news2012_short.txt.\n",
      " | Shortest len = 1.\n",
      " | Longest  len = 1090.\n",
      " | Total    sen = 624345.\n",
      "| Wrote to /home/ma-user/work/train_data/news_crawl/news2012_short_len_32.mindrecord.\n",
      " | Generate Dataset for Pre-training is done.\n",
      " | Vocabulary size: 36133.\n"
     ]
    }
   ],
   "source": [
    "\"\"\"Create News Crawl Pre-Training Dataset.\"\"\"\n",
    "import os\n",
    "from src.dataset import MonoLingualDataLoader\n",
    "from src.language_model import LooseMaskedLanguageModel\n",
    "from src.utils import Dictionary\n",
    "\n",
    "input_folder_path = './dataset/news_crawl/' # Raw corpus folder\n",
    "output_folder_path = './train_data/news_crawl/' # Dataset output path\n",
    "\n",
    "if not os.path.isdir(output_folder_path):\n",
    "    os.mkdir('./train_data')\n",
    "    os.mkdir('./train_data/news_crawl/')\n",
    "\n",
    "vocab_path = './vocab/all_en.dict.bin' # Existed vocab path\n",
    "vocab = Dictionary.load_from_persisted_dict(vocab_path)\n",
    "\n",
    "def create_pre_train(text_file, output_folder, vocab, max_sen_len):\n",
    "    \n",
    "    loader = MonoLingualDataLoader(\n",
    "        src_filepath=text_file,\n",
    "        lang=\"en\", dictionary=vocab,\n",
    "        language_model=LooseMaskedLanguageModel(mask_ratio=0.5, mask_all_prob=None),\n",
    "        max_sen_len=max_sen_len, min_sen_len=10\n",
    "    )\n",
    "\n",
    "    src_file_name = os.path.basename(text_file)\n",
    "\n",
    "    file_name = os.path.join(\n",
    "        output_folder_path,\n",
    "        src_file_name.replace('.txt', f'_len_{max_sen_len}.mindrecord')\n",
    "    )\n",
    "    \n",
    "    if os.path.exists(file_name):\n",
    "        os.remove(file_name)\n",
    "    if os.path.exists(file_name+'.db'):\n",
    "        os.remove(file_name+'.db')\n",
    "        \n",
    "    loader.write_to_mindrecord(path=file_name)\n",
    "\n",
    "for file in os.listdir(input_folder_path):\n",
    "    if file.endswith(\".txt\"):\n",
    "        create_pre_train(os.path.join(input_folder_path, file),output_folder_path, vocab, 32)\n",
    "\n",
    "print(f\" | Generate Dataset for Pre-training is done.\")\n",
    "print(f\" | Vocabulary size: {vocab.size}.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Processing corpus ./dataset/ggw_data/train_short.src.txt.\n",
      " | Processing corpus ./dataset/ggw_data/train_short.tgt.txt.\n",
      " | Shortest len = 4.\n",
      " | Longest  len = 90.\n",
      " | Total    sen = 243190.\n",
      " | Total token num=10954891, 11.153657302477953% replaced by <unk>.\n",
      "| Wrote to /home/ma-user/work/train_data/ggw_data/gigaword_train_dataset.mindrecord.\n",
      " | Processing corpus ./dataset/ggw_data/test.src.txt.\n",
      " | Processing corpus ./dataset/ggw_data/test.tgt.txt.\n",
      " | Shortest len = 2.\n",
      " | Longest  len = 73.\n",
      " | Total    sen = 1081.\n",
      " | Total token num=46933, 10.623654997549698% replaced by <unk>.\n",
      "| Wrote to /home/ma-user/work/train_data/ggw_data/gigaword_test_dataset.mindrecord.\n",
      " | Generate Dataset for fine-tuneing is done.\n",
      " | Vocabulary size: 36133.\n"
     ]
    }
   ],
   "source": [
    "\"\"\"Generate Gigaword dataset.\"\"\"\n",
    "import os\n",
    "from src.dataset import BiLingualDataLoader\n",
    "from src.language_model import NoiseChannelLanguageModel\n",
    "from src.utils import Dictionary\n",
    "\n",
    "input_folder_path = './dataset/ggw_data/'\n",
    "output_folder_path = './train_data/ggw_data/'\n",
    "\n",
    "if not os.path.isdir(output_folder_path):\n",
    "    os.mkdir('./train_data/ggw_data/')\n",
    "\n",
    "vocab_path = './vocab/all_en.dict.bin'\n",
    "vocab = Dictionary.load_from_persisted_dict(vocab_path)\n",
    "\n",
    "train = BiLingualDataLoader(\n",
    "    src_filepath=os.path.join(input_folder_path,\"train_short.src.txt\"),\n",
    "    tgt_filepath=os.path.join(input_folder_path,\"train_short.tgt.txt\"),\n",
    "    src_dict=vocab, tgt_dict=vocab,\n",
    "    src_lang=\"en\", tgt_lang=\"en\",\n",
    "    language_model=NoiseChannelLanguageModel(add_noise_prob=0),\n",
    "    max_sen_len=32\n",
    ")\n",
    "\n",
    "train_path = os.path.join(output_folder_path, \"gigaword_train_dataset.mindrecord\")\n",
    "if os.path.exists(train_path):\n",
    "        os.remove(train_path)\n",
    "if os.path.exists(train_path+'.db'):\n",
    "        os.remove(train_path+'.db')\n",
    "\n",
    "train.write_to_mindrecord(\n",
    "    path=train_path\n",
    ")\n",
    "\n",
    "test = BiLingualDataLoader(\n",
    "    src_filepath=os.path.join(input_folder_path,\"test.src.txt\"),\n",
    "    tgt_filepath=os.path.join(input_folder_path,\"test.tgt.txt\"),\n",
    "    src_dict=vocab, tgt_dict=vocab,\n",
    "    src_lang=\"en\", tgt_lang=\"en\",\n",
    "    language_model=NoiseChannelLanguageModel(add_noise_prob=0),\n",
    "    max_sen_len=32\n",
    ")\n",
    "\n",
    "test_path = os.path.join(output_folder_path, \"gigaword_test_dataset.mindrecord\")\n",
    "if os.path.exists(test_path):\n",
    "        os.remove(test_path)\n",
    "if os.path.exists(test_path+'.db'):\n",
    "        os.remove(test_path+'.db')\n",
    "        \n",
    "test.write_to_mindrecord(\n",
    "    path=test_path\n",
    ")\n",
    "\n",
    "print(f\" | Generate Dataset for fine-tuneing is done.\")\n",
    "print(f\" | Vocabulary size: {vocab.size}.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 读取数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"Dataset loader to feed into model.\"\"\"\n",
    "import mindspore as ms\n",
    "import mindspore.dataset as ds\n",
    "\n",
    "def load_dataset(input_file, batch_size: int, epoch_count: int,\n",
    "                 sink_mode: bool, sink_step: int = 1, rank_size: int = 1, rank_id: int = 0, shuffle=True):\n",
    "    \n",
    "    print(f\" | Loading {input_file}.\")\n",
    "    \n",
    "    data_set = ds.MindDataset(\n",
    "        input_file,\n",
    "        columns_list=[\n",
    "            \"src\", \"src_padding\",\n",
    "            \"prev_opt\", \"prev_padding\",\n",
    "            \"target\", \"tgt_padding\"\n",
    "        ],\n",
    "        shuffle=shuffle, num_shards=rank_size, shard_id=rank_id)\n",
    "    \n",
    "    type_cast_op = ds.transforms.transforms.TypeCast(ms.int32)\n",
    "    data_set = data_set.map(operations=type_cast_op, input_columns=\"src\")\n",
    "    data_set = data_set.map(operations=type_cast_op, input_columns=\"src_padding\")\n",
    "    data_set = data_set.map(operations=type_cast_op, input_columns=\"prev_opt\")\n",
    "    data_set = data_set.map(operations=type_cast_op, input_columns=\"prev_padding\")\n",
    "    data_set = data_set.map(operations=type_cast_op, input_columns=\"target\")\n",
    "    data_set = data_set.map(operations=type_cast_op, input_columns=\"tgt_padding\")\n",
    "    \n",
    "\n",
    "    ori_dataset_size = data_set.get_dataset_size()\n",
    "    print(f\" | Dataset size: {ori_dataset_size}.\")\n",
    "    repeat_count = epoch_count\n",
    "\n",
    "    data_set = data_set.rename(\n",
    "        input_columns=[\"src\",\n",
    "                       \"src_padding\",\n",
    "                       \"prev_opt\",\n",
    "                       \"prev_padding\",\n",
    "                       \"target\",\n",
    "                       \"tgt_padding\"],\n",
    "        output_columns=[\"source_eos_ids\",\n",
    "                        \"source_eos_mask\",\n",
    "                        \"target_sos_ids\",\n",
    "                        \"target_sos_mask\",\n",
    "                        \"target_eos_ids\",\n",
    "                        \"target_eos_mask\"]\n",
    "    )\n",
    "    \n",
    "    # apply batch operations\n",
    "    data_set = data_set.batch(batch_size, drop_remainder=True)\n",
    "    data_set = data_set.repeat(repeat_count)\n",
    "\n",
    "    data_set.channel_name = 'transformer'\n",
    "    return data_set\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 3.5 预训练\n",
    "\n",
    "预训练中，采用Adam优化器和损失放大来得到预训练后的模型。\n",
    "\n",
    "这边修改了源码/src/transformer/embedding.py第81行的embedding_table.value()\n",
    "为self.embedding_table\n",
    "和/src/transformer/transformer_for_train.py第301行的scaling_sens.value()为scaling_sens\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"定义训练与推理的参数\"\"\"\n",
    "import mindspore.common.dtype as mstype\n",
    "class config():\n",
    "    enable_modelarts = True #Whether training on modelarts, default = False\n",
    "    device_target = \"Ascend\"\n",
    "    output_path = \"./output/\"\n",
    "    save_checkpoint_path = \"./output/checkpoint/\"\n",
    "    checkpoint_file_path = \"\"\n",
    "    # Training options\n",
    "    epochs = 20\n",
    "    batch_size = 192\n",
    "    dtype = mstype.float32 #only support float16 and float32\n",
    "    compute_type = mstype.float16 #only support float16 and float32\n",
    "    pre_train_dataset = \"./train_data/news_crawl/news2012_short_len_32.mindrecord\"\n",
    "    # ./train_data/news_crawl/news2012_len_32.tfrecord-001-of-001\n",
    "    fine_tune_dataset = \"./train_data/ggw_data/gigaword_train_dataset.mindrecord\"\n",
    "    test_dataset = \"./train_data/ggw_data/gigaword_test_dataset.mindrecord\"\n",
    "    dataset_sink_mode = False\n",
    "    dataset_sink_step = 100\n",
    "    random_seed = 100\n",
    "    save_graphs = False\n",
    "    seq_length = 32 #64\n",
    "    vocab_size = 0 #need change\n",
    "    hidden_size = 1024\n",
    "    num_hidden_layers = 6\n",
    "    num_attention_heads = 8\n",
    "    intermediate_size = 4096\n",
    "    hidden_act = \"relu\"\n",
    "    hidden_dropout_prob = 0.2\n",
    "    attention_dropout_prob = 0.2\n",
    "    max_position_embeddings = 32 #64\n",
    "    initializer_range = 0.02\n",
    "    label_smoothing = 0.1\n",
    "    beam_width = 4\n",
    "    length_penalty_weight = 1.0\n",
    "    max_decode_length = 32 #64\n",
    "    init_loss_scale = 65536\n",
    "    loss_scale_factor = 2\n",
    "    scale_window = 200\n",
    "    lr = 0.0001\n",
    "    poly_lr_scheduler_power = 0.5\n",
    "    decay_steps = 10000\n",
    "    decay_start_step = 12000\n",
    "    warmup_steps = 4000\n",
    "    min_lr = 0.000001\n",
    "    save_ckpt_steps = 10000\n",
    "    keep_ckpt_max = 50\n",
    "    ckpt_prefix = \"pt\"\n",
    "    metric = \"rouge\"\n",
    "    vocab = \"./vocab/all_en.dict.bin\"\n",
    "    output = \"./output/infer.bin\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pickle\n",
    "import numpy as np\n",
    "\n",
    "from mindspore.common.tensor import Tensor\n",
    "from mindspore.nn import Momentum\n",
    "from mindspore.nn.optim import Adam, Lamb\n",
    "from mindspore.train.model import Model\n",
    "from mindspore.train.loss_scale_manager import DynamicLossScaleManager, FixedLossScaleManager\n",
    "from mindspore.train.callback import CheckpointConfig, ModelCheckpoint, TimeMonitor\n",
    "from mindspore import context, Parameter\n",
    "from mindspore.communication import management as MultiAscend\n",
    "from mindspore.train.serialization import load_checkpoint\n",
    "from mindspore.common import set_seed\n",
    "\n",
    "from src.transformer import TransformerNetworkWithLoss, TransformerTrainOneStepWithLossScaleCell\n",
    "from src.utils import LossCallBack\n",
    "from src.utils import one_weight, zero_weight, weight_variable\n",
    "from src.utils import square_root_schedule\n",
    "from src.utils.lr_scheduler import polynomial_decay_scheduler, BertLearningRate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Starting training on single device.\n",
      " | Loading ./train_data/news_crawl/news2012_short_len_32.mindrecord.\n",
      " | Dataset size: 624345.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] ME(112070:281473693656960,MainProcess):2022-11-26-16:18:32.185.302 [mindspore/train/model.py:1077] For LossCallBack callback, {'step_end'} methods may not be supported in later version, Use methods prefixed with 'on_train' or 'on_eval' instead when using customized callbacks.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Start pre-training job.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.785.371 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/dropout-Dropout/DropoutGenMask-op5144] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.785.589 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/0-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5145] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.785.749 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/0-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5146] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.785.895 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/1-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5147] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.786.038 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/1-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5148] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.786.182 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/2-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5149] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.786.332 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/2-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5150] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.786.495 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/3-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5151] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.786.640 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/3-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5152] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.786.782 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/4-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5153] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.786.919 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/4-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5154] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.787.057 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/5-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5155] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.787.200 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/5-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5156] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.787.343 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/dropout-Dropout/DropoutGenMask-op5157] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.787.481 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5158] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.787.619 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5159] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.787.758 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5160] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.787.898 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5161] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.788.043 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5162] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.788.179 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5163] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.788.320 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5164] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.788.457 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5165] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.788.598 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5166] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.788.738 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5167] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.788.893 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5168] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.789.041 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5169] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.789.181 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5170] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.789.319 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5171] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.789.456 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5172] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.789.596 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5173] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.789.739 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5174] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.789.883 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op5175] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.973.133 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/0-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5437] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.973.317 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/1-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5438] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.973.477 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/2-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5439] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.973.625 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/3-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5440] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.973.769 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/4-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5441] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:30.973.915 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/5-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5442] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.011.328 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5446] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.011.511 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5447] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.011.664 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5448] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.011.815 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5449] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.011.966 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5450] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.012.111 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op5451] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.189.179 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/0-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5718] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.189.364 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/1-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5719] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.189.513 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/2-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5720] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.189.657 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/3-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5721] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.189.801 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/4-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5722] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.189.948 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/5-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5723] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.192.443 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5726] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.192.613 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5727] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.192.775 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5728] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.192.921 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5729] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.193.066 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5730] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.193.213 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5731] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.193.356 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5732] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.193.498 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5733] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.193.639 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5734] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.193.778 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5735] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.193.921 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5736] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:19:31.194.063 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op5737] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:20:26.444.439 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_stream_assign.cc:1944] InsertEventForCallCommSubGraph] Cannot find comm group for sub comm graph label id 2\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train epoch time: 563198.464 ms, per step time: 173.239 ms\n"
     ]
    }
   ],
   "source": [
    "config = config()\n",
    "config.epochs = 1 #for test\n",
    "\n",
    "vocab_path = './vocab/all_en.dict.bin'\n",
    "vocab = Dictionary.load_from_persisted_dict(vocab_path)\n",
    "config.vocab_size = vocab.size\n",
    "\n",
    "print(\" | Starting training on single device.\")\n",
    "pre_train_dataset = load_dataset(input_file=config.pre_train_dataset,\n",
    "                                 batch_size=config.batch_size,\n",
    "                                 epoch_count=1,\n",
    "                                 sink_mode=config.dataset_sink_mode,\n",
    "                                 sink_step=config.dataset_sink_step)\n",
    "\n",
    "# 定义帶loss的網路\n",
    "net_with_loss = TransformerNetworkWithLoss(config, is_training=True)\n",
    "net_with_loss.init_parameters_data()\n",
    "\n",
    "for param in net_with_loss.trainable_params():\n",
    "    name = param.name\n",
    "    value = param.data\n",
    "    if isinstance(value, Tensor):\n",
    "        if name.endswith(\".gamma\"):\n",
    "            param.set_data(one_weight(value.asnumpy().shape))\n",
    "        elif name.endswith(\".beta\") or name.endswith(\".bias\"):\n",
    "            param.set_data(zero_weight(value.asnumpy().shape))\n",
    "        else:\n",
    "            param.set_data(weight_variable(value.asnumpy().shape))\n",
    "\n",
    "update_steps = config.epochs * pre_train_dataset.get_dataset_size()\n",
    "\n",
    "# 定义递减的学习率\n",
    "lr = Tensor(polynomial_decay_scheduler(lr=config.lr,\n",
    "           min_lr=config.min_lr,\n",
    "           decay_steps=config.decay_steps,\n",
    "           total_update_num=update_steps,\n",
    "           warmup_steps=config.warmup_steps,\n",
    "           power=config.poly_lr_scheduler_power), dtype=mstype.float32)\n",
    "# 定义优化器\n",
    "optimizer = Adam(net_with_loss.trainable_params(), lr, beta1=0.9, beta2=0.98)\n",
    "\n",
    "# loss scale (mode = dynamic)\n",
    "scale_manager = DynamicLossScaleManager(init_loss_scale=config.init_loss_scale,\n",
    "                                        scale_factor=config.loss_scale_factor,\n",
    "                                        scale_window=config.scale_window)\n",
    "# 定义反向网络\n",
    "net_with_grads = TransformerTrainOneStepWithLossScaleCell(network=net_with_loss, optimizer=optimizer,\n",
    "                                                          scale_update_cell=scale_manager.get_update_cell())\n",
    "net_with_grads.set_train(True)\n",
    "\n",
    "# 初始化模型\n",
    "model = Model(net_with_grads)\n",
    "\n",
    "time_cb = TimeMonitor(data_size=pre_train_dataset.get_dataset_size())\n",
    "ckpt_config = CheckpointConfig(save_checkpoint_steps=config.save_ckpt_steps,\n",
    "                               keep_checkpoint_max=config.keep_ckpt_max)\n",
    "\n",
    "callbacks = []\n",
    "callbacks.append(time_cb)\n",
    "ckpt_save_dir = config.save_checkpoint_path\n",
    "\n",
    "ckpt_callback = ModelCheckpoint(\n",
    "    prefix=config.ckpt_prefix,\n",
    "    directory=os.path.join(ckpt_save_dir, 'ckpt_pt'),\n",
    "    config=ckpt_config)\n",
    "loss_monitor = LossCallBack(rank_id=0)\n",
    "callbacks.append(loss_monitor)\n",
    "callbacks.append(ckpt_callback)\n",
    "\n",
    "print(\" | Start pre-training job.\")\n",
    "model.train(config.epochs, pre_train_dataset,\n",
    "            callbacks=callbacks, dataset_sink_mode=config.dataset_sink_mode,\n",
    "            sink_size=config.dataset_sink_step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6 微调\n",
    "\n",
    "微调时，根据不同的任务，采用不同的数据集对预训练的模型进行微调。\n",
    "\n",
    "这边需要手动设置下面config类别中的checkpoint_file_path参数为预训练生成的模型文件(./output/checkpoint/ckpt_pt/pt-XXX.ckpt)。<br>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"定义训练与推理的参数\"\"\"\n",
    "import mindspore.common.dtype as mstype\n",
    "class config():\n",
    "    enable_modelarts = True #Whether training on modelarts, default = False\n",
    "    device_target = \"Ascend\"\n",
    "    output_path = \"./output/\"\n",
    "    save_checkpoint_path = \"./output/checkpoint/\"\n",
    "    checkpoint_file_path = \"\"\n",
    "    # Training options\n",
    "    epochs = 20\n",
    "    batch_size = 192\n",
    "    dtype = mstype.float32 #only support float16 and float32\n",
    "    compute_type = mstype.float16 #only support float16 and float32\n",
    "    pre_train_dataset = \"./train_data/news_crawl/news2012_short_len_32.mindrecord\"\n",
    "    # ./train_data/news_crawl/news2012_len_32.tfrecord-001-of-001\n",
    "    fine_tune_dataset = \"./train_data/ggw_data/gigaword_train_dataset.mindrecord\"\n",
    "    test_dataset = \"./train_data/ggw_data/gigaword_test_dataset.mindrecord\"\n",
    "    dataset_sink_mode = False\n",
    "    dataset_sink_step = 100\n",
    "    random_seed = 100\n",
    "    save_graphs = False\n",
    "    seq_length = 32 #64\n",
    "    vocab_size = 0 #need change\n",
    "    hidden_size = 1024\n",
    "    num_hidden_layers = 6\n",
    "    num_attention_heads = 8\n",
    "    intermediate_size = 4096\n",
    "    hidden_act = \"relu\"\n",
    "    hidden_dropout_prob = 0.2\n",
    "    attention_dropout_prob = 0.2\n",
    "    max_position_embeddings = 32 #64\n",
    "    initializer_range = 0.02\n",
    "    label_smoothing = 0.1\n",
    "    beam_width = 4\n",
    "    length_penalty_weight = 1.0\n",
    "    max_decode_length = 32 #64\n",
    "    init_loss_scale = 65536\n",
    "    loss_scale_factor = 2\n",
    "    scale_window = 200\n",
    "    lr = 0.0001\n",
    "    poly_lr_scheduler_power = 0.5\n",
    "    decay_steps = 10000\n",
    "    decay_start_step = 12000\n",
    "    warmup_steps = 4000\n",
    "    min_lr = 0.000001\n",
    "    save_ckpt_steps = 10000\n",
    "    keep_ckpt_max = 50\n",
    "    ckpt_prefix = \"ft\"\n",
    "    metric = \"rouge\"\n",
    "    vocab = \"./vocab/all_en.dict.bin\"\n",
    "    output = \"./output/infer.bin\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Starting training on single device.\n",
      " | Loading ./train_data/ggw_data/gigaword_train_dataset.mindrecord.\n",
      " | Dataset size: 243190.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] ME(112070:281473693656960,MainProcess):2022-11-26-16:31:39.976.975 [mindspore/train/model.py:1077] For LossCallBack callback, {'step_end'} methods may not be supported in later version, Use methods prefixed with 'on_train' or 'on_eval' instead when using customized callbacks.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Start fine-tuning job.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.728.877 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/dropout-Dropout/DropoutGenMask-op17463] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.729.128 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/0-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17464] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.729.315 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/0-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17465] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.729.488 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/1-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17466] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.729.670 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/1-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17467] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.729.859 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/2-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17468] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.730.044 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/2-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17469] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.730.236 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/3-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17470] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.730.431 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/3-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17471] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.730.613 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/4-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17472] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.730.801 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/4-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17473] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.730.979 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/5-EncoderCell/attention-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17474] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.731.166 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/5-EncoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17475] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.731.342 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/dropout-Dropout/DropoutGenMask-op17476] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.731.509 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17477] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.731.670 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17478] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.731.831 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17479] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.732.003 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17480] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.732.173 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17481] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.732.344 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17482] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.732.517 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17483] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.732.695 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17484] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.732.857 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17485] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.733.025 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17486] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.733.194 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17487] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.733.357 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17488] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.733.529 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17489] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.733.694 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17490] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.733.856 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17491] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.734.018 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/masked_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17492] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.734.183 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/enc_dec_attn-SelfAttention/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17493] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.734.370 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/feed_forward_net-FeedForwardNet/residual-ResidualConnection/dropout-Dropout/DropoutGenMask-op17494] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.945.367 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/0-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17756] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.945.593 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/1-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17757] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.945.778 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/2-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17758] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.945.945 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/3-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17759] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.946.107 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/4-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17760] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.946.278 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/5-EncoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17761] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.988.771 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17765] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.989.010 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17766] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.989.196 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17767] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.989.378 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17768] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.989.549 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17769] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:33.989.717 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/feed_forward_net-FeedForwardNet/dropout-Dropout/DropoutGenMask-op17770] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.193.415 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/0-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18037] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.193.658 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/1-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18038] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.193.842 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/2-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18039] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.194.016 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/3-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18040] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.194.180 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/4-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18041] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.194.380 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/encoder-TransformerEncoder/layers-CellList/5-EncoderCell/attention-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18042] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.197.325 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18045] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.197.546 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer0-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18046] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.197.735 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18047] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.197.912 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer1-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18048] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.198.080 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18049] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.198.260 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer2-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18050] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.198.459 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18051] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.198.637 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer3-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18052] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.198.798 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18053] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.198.975 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer4-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18054] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.199.142 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/masked_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18055] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:32:34.199.310 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerNetworkWithLoss/transformer-TransformerTraining/transformer-Transformer/decoder-TransformerDecoder/layer5-DecoderCell/enc_dec_attn-SelfAttention/multi_head_self_attention-MultiHeadAttention/dropout-Dropout/DropoutGenMask-op18056] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:33:15.590.395 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_stream_assign.cc:1944] InsertEventForCallCommSubGraph] Cannot find comm group for sub comm graph label id 2\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train epoch time: 275853.593 ms, per step time: 217.894 ms\n"
     ]
    }
   ],
   "source": [
    "config = config()\n",
    "# config.checkpoint_file_path = './output/checkpoint/ckpt_pt/pt-1_3251.ckpt' #赋值给预训练生成的已有模型文件\n",
    "config.epochs = 1 #for test\n",
    "\n",
    "vocab_path = './vocab/all_en.dict.bin'\n",
    "vocab = Dictionary.load_from_persisted_dict(vocab_path)\n",
    "config.vocab_size = vocab.size\n",
    "\n",
    "print(\" | Starting training on single device.\")\n",
    "fine_tune_dataset = load_dataset(input_file=config.fine_tune_dataset,\n",
    "                                 batch_size=config.batch_size,\n",
    "                                 epoch_count=1,\n",
    "                                 sink_mode=config.dataset_sink_mode,\n",
    "                                 sink_step=config.dataset_sink_step)\n",
    "\n",
    "# 定义帶loss的網路\n",
    "net_with_loss = TransformerNetworkWithLoss(config, is_training=True)\n",
    "net_with_loss.init_parameters_data()\n",
    "\n",
    "# 读取已有模型文件的权重\n",
    "weights = load_checkpoint(config.checkpoint_file_path)\n",
    "for param in net_with_loss.trainable_params():\n",
    "    weights_name = param.name\n",
    "    if isinstance(weights[weights_name], Parameter):\n",
    "        param.set_data(weights[weights_name].data)\n",
    "    elif isinstance(weights[weights_name], Tensor):\n",
    "        param.set_data(Tensor(weights[weights_name].asnumpy(), config.dtype))\n",
    "    elif isinstance(weights[weights_name], np.ndarray):\n",
    "        param.set_data(Tensor(weights[weights_name], config.dtype))\n",
    "    else:\n",
    "        param.set_data(weights[weights_name])\n",
    "\n",
    "update_steps = config.epochs * fine_tune_dataset.get_dataset_size()\n",
    "\n",
    "# 定义递减的学习率\n",
    "lr = Tensor(polynomial_decay_scheduler(lr=config.lr,\n",
    "           min_lr=config.min_lr,\n",
    "           decay_steps=config.decay_steps,\n",
    "           total_update_num=update_steps,\n",
    "           warmup_steps=config.warmup_steps,\n",
    "           power=config.poly_lr_scheduler_power), dtype=mstype.float32)\n",
    "# 定义优化器\n",
    "optimizer = Adam(net_with_loss.trainable_params(), lr, beta1=0.9, beta2=0.98)\n",
    "\n",
    "# loss scale (mode = dynamic)\n",
    "scale_manager = DynamicLossScaleManager(init_loss_scale=config.init_loss_scale,\n",
    "                                        scale_factor=config.loss_scale_factor,\n",
    "                                        scale_window=config.scale_window)\n",
    "# 定义反向网络\n",
    "net_with_grads = TransformerTrainOneStepWithLossScaleCell(network=net_with_loss, optimizer=optimizer,\n",
    "                                                          scale_update_cell=scale_manager.get_update_cell())\n",
    "net_with_grads.set_train(True)\n",
    "\n",
    "# 初始化模型\n",
    "model = Model(net_with_grads)\n",
    "\n",
    "time_cb = TimeMonitor(data_size=fine_tune_dataset.get_dataset_size())\n",
    "ckpt_config = CheckpointConfig(save_checkpoint_steps=config.save_ckpt_steps,\n",
    "                               keep_checkpoint_max=config.keep_ckpt_max)\n",
    "\n",
    "callbacks = []\n",
    "callbacks.append(time_cb)\n",
    "ckpt_save_dir = config.save_checkpoint_path\n",
    "\n",
    "ckpt_callback = ModelCheckpoint(\n",
    "    prefix=config.ckpt_prefix,\n",
    "    directory=os.path.join(ckpt_save_dir, 'ckpt_ft'),\n",
    "    config=ckpt_config)\n",
    "loss_monitor = LossCallBack(rank_id=0)\n",
    "callbacks.append(loss_monitor)\n",
    "callbacks.append(ckpt_callback)\n",
    "\n",
    "print(\" | Start fine-tuning job.\")\n",
    "model.train(config.epochs, fine_tune_dataset,\n",
    "            callbacks=callbacks, dataset_sink_mode=config.dataset_sink_mode,\n",
    "            sink_size=config.dataset_sink_step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.7 推理\n",
    "测试过程中，通过微调后的模型预测结果，并采用波束大小為4的搜索算法获取可能性最高的预测结果。\n",
    "\n",
    "这边需要手动设置下面config类别中的checkpoint_file_path参数为微调生成的模型文件(./output/checkpoint/ckpt_ft/ft-XXX.ckpt)。<br>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"定义训练与推理的参数\"\"\"\n",
    "import mindspore.common.dtype as mstype\n",
    "class config():\n",
    "    enable_modelarts = True #Whether training on modelarts, default = False\n",
    "    device_target = \"Ascend\"\n",
    "    output_path = \"./output/\"\n",
    "    save_checkpoint_path = \"./output/checkpoint/\"\n",
    "    checkpoint_file_path = \"\"\n",
    "    # Training options\n",
    "    epochs = 20\n",
    "    batch_size = 192\n",
    "    dtype = mstype.float32 #only support float16 and float32\n",
    "    compute_type = mstype.float16 #only support float16 and float32\n",
    "    pre_train_dataset = \"./train_data/news_crawl/news2012_short_len_32.mindrecord\"\n",
    "    # ./train_data/news_crawl/news2012_len_32.tfrecord-001-of-001\n",
    "    fine_tune_dataset = \"./train_data/ggw_data/gigaword_train_dataset.mindrecord\"\n",
    "    test_dataset = \"./train_data/ggw_data/gigaword_test_dataset.mindrecord\"\n",
    "    dataset_sink_mode = False\n",
    "    dataset_sink_step = 100\n",
    "    random_seed = 100\n",
    "    save_graphs = False\n",
    "    seq_length = 32 #64\n",
    "    vocab_size = 0 #need change\n",
    "    hidden_size = 1024\n",
    "    num_hidden_layers = 6\n",
    "    num_attention_heads = 8\n",
    "    intermediate_size = 4096\n",
    "    hidden_act = \"relu\"\n",
    "    hidden_dropout_prob = 0.2\n",
    "    attention_dropout_prob = 0.2\n",
    "    max_position_embeddings = 32 #64\n",
    "    initializer_range = 0.02\n",
    "    label_smoothing = 0.1\n",
    "    beam_width = 4\n",
    "    length_penalty_weight = 1.0\n",
    "    max_decode_length = 32 #64\n",
    "    init_loss_scale = 65536\n",
    "    loss_scale_factor = 2\n",
    "    scale_window = 200\n",
    "    lr = 0.0001\n",
    "    poly_lr_scheduler_power = 0.5\n",
    "    decay_steps = 10000\n",
    "    decay_start_step = 12000\n",
    "    warmup_steps = 4000\n",
    "    min_lr = 0.000001\n",
    "    save_ckpt_steps = 10000\n",
    "    keep_ckpt_max = 50\n",
    "    ckpt_prefix = \"ft\"\n",
    "    metric = \"rouge\"\n",
    "    vocab = \"./vocab/all_en.dict.bin\"\n",
    "    output = \"./output/infer.bin\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Loading ./train_data/ggw_data/gigaword_test_dataset.mindrecord.\n",
      " | Dataset size: 1081.\n",
      " | Load weights successfully.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:23.370.979 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op49409] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:27.029.813 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op49571] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:28.568.189 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op49733] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:30.250.642 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op49895] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:31.824.302 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op50057] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:33.410.887 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op50219] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:35.037.489 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op50381] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:39.360.820 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op50543] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:41.003.569 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op50705] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:42.616.473 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op50867] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:44.141.875 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op51029] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:45.681.768 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op51191] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:47.261.896 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op51353] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:48.825.720 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op51515] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:50.409.080 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op51677] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:55.747.606 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op51839] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:57.317.285 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op52001] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:49:58.832.362 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op52163] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:00.342.412 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op52325] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:01.904.841 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op52487] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:03.416.348 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op52649] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:04.957.332 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op52811] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:06.504.024 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op52973] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:07.991.011 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op53135] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:09.500.082 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op53297] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:14.733.837 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op53459] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:16.353.042 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op53621] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:17.952.913 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op53783] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:19.517.561 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op53945] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:21.050.794 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op54107] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:22.637.991 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op54269] don't support int64, reduce precision from int64 to int32.\n",
      "[WARNING] DEVICE(112070,ffffb3862780,python):2022-11-26-16:50:24.230.913 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:330] FilterRaisedOrReducePrecisionMatchedKernelInfo] Operator:[Default/network-TransformerInferModel/decoder-BeamSearchDecoder/TopK-op54431] don't support int64, reduce precision from int64 to int32.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " | Batch size: 192, Time cost: 302.5279002189636.\n",
      " | Batch size: 192, Time cost: 3.768460273742676.\n",
      " | Batch size: 192, Time cost: 3.782809257507324.\n",
      " | Batch size: 192, Time cost: 3.8514673709869385.\n",
      " | Batch size: 192, Time cost: 3.8059146404266357.\n",
      " | source: japan 's nec corp. and UNK computer corp. of the united states said wednesday they had agreed to join forces in <unk> sales . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: nec UNK in computer sales tie-up </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the sri <unk> government on wednesday announced the closure of government schools with immediate effect as a military campaign against <unk> separatists escalated in the north of the country . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: sri lanka closes schools as war escalates </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: police arrested five anti-nuclear protesters thursday after they sought to disrupt loading of a french antarctic research and supply vessel , a spokesman for the protesters said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: protesters target french research ship </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: factory orders for manufactured goods rose <unk> percent in <unk> , the commerce department said here thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us <unk> factory orders up <unk> percent </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the bank of japan appealed to financial markets to remain calm friday following the us decision to order <unk> bank <unk> to close its us operations . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bank of UNK UNK for calm in financial markets </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russian liberal party wins registration </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: turnout was heavy for parliamentary elections monday in <unk> and <unk> after a month of intensive campaigning throughout the country , one of the most prosperous in the caribbean . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> and <unk> poll draws heavy turnout by john <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: jordan 's crown prince hassan ibn <unk> arrived <unk> for his first visit to <unk> and was to pay his condolences to the widow of assassinated prime minister <unk> rabin . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: jordan 's crown prince makes first visit to <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's main opposition party <unk> endorsed president lech <unk> in an upcoming presidential run-off election after a reformed communist won the first round of voting . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> receives opposition boost </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the rand gained ground against the dollar at the opening here wednesday , to <unk> <unk> <unk> to the greenback from <unk> <unk> <unk> at the close <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: rand gains ground </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hong kong signed a breakthrough air services agreement with the united states on friday that will allow us airlines to carry freight to asian destinations via the territory . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hong kong us sign breakthrough aviation pact </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a us citizen who spied for communist east germany was given a suspended jail sentence of <unk> months here friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us citizen who spied for east <unk> given suspended sentence </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: france , still high after their convincing <unk> win over new zealand have named the same team for the second test next saturday in paris . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: french keep same team for <unk> test </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> quarter : chi - <unk> , <unk> <unk> <unk> UNK pass from erik <unk> <unk> kevin butler kick <unk> , <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chicago # <unk> # # <unk> green bay <unk> # # # <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: at least <unk> people were killed when a nigeria airways airliner crashed on landing monday at <unk> airport in the north of the country , airport officials said . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> plane crashes on landing killing at least <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> , the former south korean military UNK charged with corruption on thursday , oversaw the first tentative steps of his country 's transition from authoritarian rule to democracy . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> coup <unk> then democracy convert now in disgrace by UNK park </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a court here thursday sentenced a <unk> man to <unk> years in jail after he admitted <unk> his baby son to death to silence him while watching television . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: man who killed baby to hear television better gets <unk> years </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: five east <unk> youths who scaled the french embassy 's fence here thursday , left the embassy on their way to portugal friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK latest east <unk> asylum seekers leave for portugal </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the repatriation of at least <unk> <unk> <unk> was postponed friday after the <unk> pulled out of the first joint scheme to return refugees to their homes in northwest <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: repatriation of <unk> refugees postponed </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: outgoing polish president lech <unk> and his ex-communist rival <unk> <unk> cast their ballots sunday in a tight contest for the presidency but managed some humor amid all the tension . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> cast ballot amid touch of humor </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: tea scores on the fourth day of the second test between australia and <unk> here monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: australia vs <unk> tea scorecard </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the head of the <unk> government in the breakaway republic of <unk> narrowly survived a bomb attack monday , the third assassination attempt against top russian officials in two months . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: head of UNK <unk> government survives bomb </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: former israeli chief of staff <unk> <unk> is to be appointed foreign minister in the new cabinet of prime minister <unk> peres , an outgoing minister said <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> to be named israel 's new fm outgoing minister says </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: president <unk> <unk> in nigeria </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> prime minister <unk> <unk> indicated friday he would soon relinquish control of the ruling party to his deputy anwar <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> wants leadership change to be smooth </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ministers from the european union and its <unk> neighbors gathered here under heavy security on monday for an unprecedented conference on economic and political cooperation . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: european <unk> ministers gather for landmark conference by <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> telecommunications giant <unk> has reached a basic agreement to sell its relay production to <unk> electronics company UNK corp , <unk> said thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> sells relay production to UNK 's UNK corp </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russian foreign minister <unk> <unk> on thursday welcomed a us move to enter <unk> talks on iran 's nuclear program , urging the international community not to disrupt the process . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russia urges international community not to disrupt iran talks UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: football fans in the <unk> capital <unk> will be able to watch the world cup live on big screens at <unk> city locations , officials said friday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: the <unk> world news summary </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> won the toss and chose to bat on the opening day in the opening test against west indies at the <unk> recreation ground on friday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> win toss and elect to bat in first test </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: polling stations closed at <unk> <unk> <unk> <unk> local time <unk> friday on the first day of <unk> legislative elections <unk> by allegations surrounding social democrat prime minister <unk> <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: polling stations close on first day of <unk> legislative elections </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an <unk> indian state is paying handsome rewards to policemen for slaying <unk> rebels in the <unk> despite protests from rights groups about <unk> violence . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: police turn UNK in <unk> indian state by UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the united nations condemned saturday an attack on russian embassy employees in <unk> that claimed the life of one russian and resulted in the kidnapping of four others . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: un condemns murder UNK of <unk> in iraq UNK UNK UNK with annan comment </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: american time-trial specialist david <unk> of team <unk> won the <unk> km opening prologue of the <unk> <unk> here on sunday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: american <unk> <unk> <unk> prologue </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a powerful bomb exploded outside a navy base near the sri <unk> capital <unk> <unk> , seriously wounding at least one person , military officials said . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bomb attack outside <unk> navy base </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chelsea manager jose <unk> has said that only another title and the champions league will <unk> for him as he builds towards next season . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> targets second champions league crown UNK UNK details </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russia warned wednesday against nato taking in the <unk> republics of <unk> and <unk> , saying such a colossal geopolitical shift would threaten relations . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russia warns of colossal impact if nato takes in <unk> <unk> UNK UNK quote </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> steel said on wednesday that its hostile takeover offer to shareholders in european steelmaker <unk> had officially begun in the united states and would last until <unk> # . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> launches hostile <unk> bid in us </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: americans have to wait probably another year before they can relish <unk> indian <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: time not ripe yet for indian <unk> to hit us </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: earthquake survivors on <unk> 's java island have endured <unk> suffering but some say they are now facing the final straw : missing out on watching the world cup . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> quake survivors brace for another hardship : no world cup by <unk> UNK UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: even some of his rivals in the roland <unk> <unk> are hoping that roger <unk> can create a bit of tennis history by winning all four grand slams . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: even fellow players keen for <unk> grand slam dream UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: question marks over the form of spain 's record <unk> captain <unk> continued to feed the concerns of the country 's press here friday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: pressure on <unk> as spain begin countdown by justin davis UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: group of eight finance ministers warned saturday that the world economy was <unk> by soaring energy costs and called for closer cooperation to calm volatile oil and gas markets . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> ministers warn of energy risks seek cooperative action by <unk> <unk> UNK picture UNK UNK with final statement </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: england coach <unk> <unk> believes his team must improve on their disappointing display against <unk> if they want to win the world cup . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: we must improve : <unk> by martin <unk> UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president <unk> <unk> needs to do more to disarm militant groups , israeli prime minister <unk> <unk> said sunday , pledging to help in the process . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: israel pm says <unk> must do more to disarm militants UNK UNK detail UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: group h favorites spain could be given an unexpected boost ahead of their opener against <unk> - if <unk> <unk> decides to sit out wednesday 's afternoon match in <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: doubts over <unk> and <unk> set to favor spain by justin davis UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: tomas <unk> scored twice as the <unk> republic enjoyed an impressive <unk> victory over the united states in their world cup opener on monday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> double gives <unk> winning start by jim <unk> UNK picture UNK UNK <unk> and arena UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the open square in front of the french capital 's famed <unk> <unk> cathedral will be renamed after late pope john paul ii , the city council decided <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> cathedral square to be named after john paul ii </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the french press on wednesday were typically scathing in their assessment of les <unk> ' performance in their opening world cup match , a <unk> draw with <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: press <unk> sorry french display </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: press freedom in <unk> remains at risk despite the release on wednesday of prominent newspaper editor <unk> UNK after a two-year prison sentence , human rights organizations said . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> press freedom at risk despite editor 's release UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: former italian prime minister silvio <unk> has been re-elected as president of serie a side ac <unk> , the club confirmed on thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> re-elected ac <unk> president </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: one of <unk> 's hottest actors john <unk> has refused to bow to pressure from <unk> 's health minister and quit promoting <unk> drinks . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> at government over <unk> drink ad comments UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: european union leaders gave <unk> a green light friday to join the eurozone next year , launching a new wave of expansion for the currently <unk> single currency club . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eu gives <unk> green light to join eurozone by <unk> thomas UNK picture UNK UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's ruling military leaders have launched an electoral campaign in support of a constitutional referendum set for <unk> <unk> , the official media announced friday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's junta campaigns for constitutional referendum </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> nations sent japan crashing to a third straight defeat on saturday at world whale talks soured by rows over cruelty and the moratorium on commercial hunts . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: fierce rows rock world whaling talks by <unk> UNK UNK UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: result in a world cup group g match here on sunday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: world cup : france # south korea # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> on ivory coast who play <unk> and <unk> in a world cup group c match here on wednesday : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: world cup <unk> <unk> : ivory coast </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: germany are renowned as masters in the penalty shootout but manager <unk> <unk> revealed monday that the world cup hosts have not been <unk> UNK in training . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: germany not <unk> pens says <unk> UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> <unk> arabia in their crunch group h game here monday , firmly setting their <unk> world cup campaign back on track with a convincing <unk> win . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> <unk> arabia for vital world cup win by <unk> <unk> UNK picture UNK UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> on croatia who face australia in a world cup group f match here on thursday : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: world cup <unk> <unk> : croatia </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> will miss their traditional <unk> <unk> in state-owned forests this year after the authorities on <unk> banned the practice because of risk of forest fires . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> fest <unk> banned in <unk> forests </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: journalists and legislators in afghanistan have been outraged by the intelligence service 's new media guidelines that bar interviews with taliban leaders and criticism of foreign troops . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: alarm over afghan guidelines for journalists by UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: south korea said wednesday it would consider scrapping food aid to impoverished north korea if <unk> <unk> a missile . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> could UNK food aid if <unk> fires missile UNK UNK aid details minister quote </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: around <unk> clandestine immigrants wednesday staged a peaceful breakout from a detention center in malta and demonstrated on a road shouting `` we want freedom . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mass breakout of immigrants from malta center </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: like a <unk> army , south korea are marching towards the <unk> stages of the <unk> world cup on their <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: team korea marching on their <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hong kong flag carrier <unk> pacific said thursday it has ordered six <unk> UNK freighter aircraft in a billion dollar deal for delivery between may <unk> and april <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> orders six <unk> <unk> freighter planes UNK UNK details UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: fbi agents have arrested at least seven people in <unk> who reportedly plotted to attack the chicago <unk> tower <unk> , according to official statements and news reports . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: fbi arrests seven in UNK plot on chicago <unk> tower UNK correction please read <unk> seas of david <unk> sted of cs of david in UNK <unk> </s> <pad> <pad> <pad> <pad>\n",
      " | source: world cup hopefuls spain secured their objective of finishing top of group h with a <unk> <unk> win over whipping boys <unk> arabia here on friday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: fans feel the pain as spain beat <unk> by justin davis UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: teenager <unk> <unk> claimed her first <unk> title with a <unk> , <unk> victory over third seed <unk> <unk> at home in the <unk> UNK open on saturday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> defeats <unk> in <unk> final UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> man hanged himself in <unk> sunday after the asian champions failed to secure a single victory in the world cup , a report said . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK fan hangs himself for nation 's dismal world cup performance : report </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: france beat south africa <unk> here on sunday to become the world <unk> rugby champions for the first time . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: france crowned world <unk> rugby champions UNK UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: china has executed five people who had been sentenced to death for drugs trafficking , state media reported on monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: china says five executed for drugs </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> moved to consolidate their premiership status <unk> by tying down one of the brightest stars of last season 's maiden top flight campaign . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> tie up <unk> for three years </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a controversial civilian nuclear energy deal between <unk> and the united states cleared its first major hurdle <unk> , easily winning approval by a us congress committee . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us congress committee backs nuclear deal with <unk> UNK UNK UNK UNK detail </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chinese government departments lost <unk> billion dollars last year through corruption , poor tax methods and bad land management , the national audit office said in a report published wednesday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: china audit finds widespread corruption tax losses in government UNK UNK report details budget UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> martyrs brigades said wednesday it had kidnapped an israeli settler , marking the third abduction of an israeli since sunday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> claims abduction of third israeli UNK UNK <unk> statement </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: greece said thursday that it was monitoring the escalating hostilities between <unk> and <unk> with `` particular concern , '' and feared it could worsen . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: greece worried over escalation of <unk> hostilities </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: european stock markets advanced strongly thursday on some bargain-hunting and gains by wall street and <unk> shares ahead of an expected hike in us interest rates , dealers said . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: european stocks bounce back UNK UNK with closing levels </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> prime minister <unk> <unk> insisted friday that his <unk> government was continuing efforts to secure the release of an israeli soldier captured by militants . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: efforts still underway to secure soldier 's release : hamas pm </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the united states deported to <unk> on friday two <unk> <unk> wanted by a local court on charges of genocide committed in the <unk> <unk> massacre , an official said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us deports to <unk> two <unk> wanted for <unk> genocide </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: luca <unk> scored twice as italy beat <unk> <unk> in the world cup quarter-finals here on friday to set up a <unk> last four clash against hosts germany . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: terrific <unk> seals semi-final place for italy by <unk> UNK UNK picture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: up to <unk> revellers packed <unk> 's streets to celebrate the gay and lesbian <unk> gras , breaking attendance records for the eye - popping annual street festival . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's gay and lesbian <unk> gras draws record-breaking crowd UNK pictures </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the head of <unk> , the top sunni <unk> authority , will visit germany next week to discuss the teaching of <unk> in europe , his spokesman said monday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: top <unk> cleric to visit germany </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it 's official . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> presents new love to the world </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: senior magistrates in france have accused president <unk> <unk> and the government of ignoring their objections when making important judicial appointments , it was reported wednesday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: france : top magistrates say government ignoring their views </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> chief <unk> <unk> wednesday told <unk> to prepare for a second wave of economic reforms to catch up with asian tigers . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> should prepare for second wave of economic reforms : <unk> chief </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's congress <unk> i <unk> party thursday said it would block some of the government 's UNK tax cut plans . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's congress party threatens to sink budget tax plans </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the united states underscored thursday its support for international efforts to contain the <unk> in <unk> and pressed president <unk> <unk> to find a compromise to end the unrest . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us backs <unk> on <unk> crisis by andre UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the us economy created <unk> jobs in february when the unemployment rate dropped from <unk> percent to <unk> percent , the government reported friday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK us economy creates <unk> jobs in UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: six us tourists and their costa rican pilot cheated death friday when their <unk> plane crashed into a <unk> <unk> outside costa rica 's capital , red cross officials said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: all six us tourists and pilot survive costa rica crash </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> rebels , led by <unk> <unk> , on saturday rejected calls by the united nations for a ceasefire , saying it could only be called after talks with <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> rebels reject un call for ceasefire </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: south korean union leaders on sunday threatened renewed strikes against new labor rules drafted by parliament to replace a controversial bill which sparked industrial unrest last december . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> unions threaten new strikes against revised labor law by c. <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: egyptian president <unk> mubarak said sunday said he would ask us president bill clinton to `` use his influence '' to halt a controversial planned <unk> settlement in east <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mubarak to ask clinton for help in halting <unk> housing plan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the rand firmed further against the dollar in early trade here <unk> , opening at <unk> <unk> <unk> to the greenback against <unk> <unk> <unk> at close on monday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: rand firms against dollar </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the arab group in the united nations on <unk> dismissed israel 's warning that a <unk> bid to apply international pressure against israel could freeze the peace process . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: arab group condemns israeli ministers remarks on peace process </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the un general assembly appeared poised to adopt a resolution denouncing israel 's plans to build housing for thousands of <unk> settlers in east <unk> . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: israel has few allies in un debate over east <unk> expansion by rene UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: thai share prices dropped <unk> percent in thin trading thursday amid profit-taking by local traders , analysts said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: thai share prices drop <unk> percent </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK <unk> break out of <unk> jail </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an UNK number of chinese legislators on friday voted against the government 's annual report on efforts to combat crime and corruption . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK chinese parliament gives rough ride to anti-crime reports </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a man has been shot dead in a catholic area of west <unk> , police said friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK man shot dead in catholic west <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a fairly strong earthquake measuring <unk> on the <unk> scale rocked wide areas of central and western japan sunday , followed by four aftershocks , the meteorological agency said . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: quake shakes wide areas in UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: japan 's finance ministry was unlikely to sell down its holding in <unk> telegraph and telephone co. . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK government unlikely to sell <unk> shares before <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a truck carrying illegal north african immigrants flipped over in northeastern spain , killing <unk> and injuring six others , police said monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eleven immigrants killed in road accident in spain </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: un secretary general <unk> annan on monday <unk> the advice of the leading industrialised nations and announced the merger of the world body 's three economic development departments into one . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: un chief <unk> <unk> <unk> economic development departments </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> share prices closed <unk> percent lower <unk> amid selling pressure on heavyweight stocks , brokers said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> shares close <unk> percent lower </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> <unk> cleric living in iran has threatened to call for a holy war against the <unk> government if it sentences to death alleged <unk> militants . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> cleric warns of holy war against <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> has put five cities on a security alert after religious unrest involving <unk> and <unk> in the northern city of <unk> , an informed source said wednesday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> puts five cities on security alert after religious unrest </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: politicians were urged thursday to regard public opinion not personal views when voting on a bill aimed at overturning the world 's first legislation allowing voluntary euthanasia . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> euthanasia widow goes public in plea for law to remain </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: delta air lines will hold a press conference at <unk> <unk> thursday to announce a major new aircraft purchase , the company said in a statement . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: delta set to announce an important order </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: vice president al gore thursday welcomed <unk> group 's admission that smoking causes cancer and its decision to help state officials sue fellow cigarette makers . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: gore welcomes <unk> admission that smoking causes cancer UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president <unk> <unk> threatened to dissolve parliament friday as deputies continued blocking the passage of the <unk> budget . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK <unk> president threatens to dissolve parliament </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new research suggested saturday that ugly suspects with `` <unk> criminal features '' are more likely to be found guilty than good - looking defendants . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: the beautiful go free while the ugly are <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the indian elephant will become extinct unless the government launches an all-out war against rampant poaching , wildlife experts say . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: indian elephant may be facing extinction : experts by UNK <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a girl of <unk> cut off a <unk> <unk> 's penis after he made advances to her , police here said sunday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: spanish girl cuts off man 's penis for making sexual advances </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a joint venture involving <unk> firms <unk> corp. and <unk> <unk> co. . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> UNK of UNK in <unk> aluminum joint venture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president bill clinton will meet with jordan 's king hussein next week to discuss the latest developments in the <unk> peace process , the white house announced monday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: clinton to meet with hussein UNK # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> prosecutors raided the world 's biggest securities house , nomura securities co. . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK prosecutors raid world 's biggest broker nomura by tim UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: china celebrated a groundbreaking visit by us vice president al gore on <unk> by signing <unk> billion dollars worth of contracts with american giants <unk> and general motors . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: china marks gore visit with <unk> billion dollar contracts hk deal by UNK holland </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: wednesday 's evening rubber prices in <unk> cents per <unk> provided by the <unk> commodity exchange : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: evening rubber prices </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> national suspected in last year 's bombing of a us barracks in <unk> arabia appeared at a crucial deportation hearing here wednesday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new charges brought against <unk> suspect in terrorist bombing </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> opposition said here thursday it had killed more than <unk> government soldiers in an ambush in the east of the country . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> opposition says <unk> government troops killed in ambush </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> denied thursday it was freezing new bilateral dealings with <unk> , but conflicting signals rocked financial markets and <unk> confusion on the diplomatic front . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> denies freeze with UNK but confusion rocks markets by <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> graphic <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> die in well organized mass suicide in california : police </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> convicted of a series of murders and two other <unk> found guilty of drug trafficking were beheaded on friday , the <unk> interior ministry said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: three beheaded in <unk> arabia </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the first southeast asian <unk> film festival opened here saturday with organizers hailing the event as a step toward rebuilding <unk> 's shattered film industry and promoting regional movies . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: southeast asian film festival opens in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> police beat an israeli photographer working for <unk> <unk> in the west bank city of <unk> on sunday and confiscated his film during a <unk> protest . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> police beat israeli UNK in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a key mediator in the <unk> hostage crisis hinted the leftist rebels holding <unk> <unk> here were seeking to negotiate a final peace accord with the government . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: key mediator hints <unk> rebels seek global peace accord by UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> and <unk> ended four days of official talks here monday , the first between the two neighbors for three years , and vowed to continue working for closer ties . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK <unk> <unk> vow to continue talks after <unk> silence UNK UNK fixing date </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: israeli prime minister <unk> <unk> may be in washington next week and could meet with president bill clinton on the crumbling <unk> peace process , the white house said monday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> may come to washington as peace process reaches critical phase </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: members of the youth wing of <unk> putin 's united russia party on saturday launched a campaign against illegal immigration under the theme : `` our money for our people . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : <unk> league results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: several hundred people from france and other countries gathered in central paris sunday to mark what they called the first `` global day for the right to die in dignity . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK militants hold global <unk> day in paris </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> were <unk> at tea in their second innings on the fifth and final day of the third test against australia at the <unk> shah <unk> stadium here on sunday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : <unk> vs australia third test scoreboard </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: there is little appetite among european union member states for military intervention in the democratic republic of <unk> , the eu 's most senior military figure warned monday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: little eu will for tough <unk> operation : military chief </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hong kong shares closed <unk> percent higher monday , led by property developers and hsbc , dealers said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hong kong shares close <unk> percent higher </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: un chief ban ki-moon said he would personally mediate in the crisis in the democratic republic of <unk> where new clashes broke out <unk> , threatening a week-old ceasefire . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ban to lead <unk> mediation as new clashes threaten truce </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> share prices opened slightly lower <unk> with profit taking emerging as the market hovered close to the key <unk> mark , dealers said . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> shares open slightly lower </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ten senior french officers filed suit against <unk> for slander on <unk> after a justice minister 's report accused them of taking part in the <unk> genocide . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ten french officers file suit over <unk> genocide report </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: retailers in the <unk> countries using the euro saw demand slip less than expected in <unk> despite the worst financial crisis for generations , official eu data showed on wednesday UNK </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eurozone retail sales slip less than expected : eu data </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: obama wins us presidency : networks UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: flash UNK UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the united states has further restricted iran 's access to the us financial system by banning certain types of fund transfers , the treasury department said thursday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK UNK UNK UNK us tightens <unk> on iran 's access to banks </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: australia coach <unk> <unk> has made eight changes to the side that was defeated by new zealand at the weekend ahead of the wallabies ' test against italy here sunday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> : <unk> rings changes for <unk> <unk> test </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> bank <unk> international trimmed its <unk> profit forecast on thursday , citing a `` changed market situation . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> UNK advisory </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> were <unk> in their first innings at lunch on the opening day of the fourth and final test against australia here on thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : <unk> <unk> at lunch in final australia test </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: thousands of people attacked chinese police in the southern city of <unk> from friday afternoon to early saturday morning , state media reported . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: thousands attack police in southern china : state media </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: us president george <unk> bush on friday warned that it would take time for federal stimulus measures to have their full economic effect amid news of steep job losses . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: it will take time for economic measures to take effect : bush </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: five french <unk> and <unk> were among six foreigners killed during the autumn climbing season in <unk> , the french embassy and <unk> officials said friday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> climbing season claims five french lives </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> foreign minister <unk> smith sunday congratulated new zealand 's new prime minister-elect john key as he praised ousted leader helen clark as a `` <unk> '' and respected politician . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: time caught up with nz 's <unk> clark says <unk> fm </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: syria and <unk> decided on monday to boost border controls and anti-terrorism coordination , as the two neighbors took a new step to strengthen ties since diplomatic relations were established . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: syria and <unk> to boost border anti-terror controls </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: fourteen men accused of membership of organised crime gangs and involvement in a savage <unk> <unk> go on trial wednesday at UNK in southern italy . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: trial opens wednesday in case of italian organised crime <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russia 's defense industry has been badly hit by the global financial crisis , deputy prime minister <unk> <unk> , a former defense minister , said <unk> . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russia 's defense sector hit by financial crisis : govt official </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: grim overall economic and corporate news pushed global stocks into reverse on <unk> despite an unexpected beam of light for the german economy . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: grim economic outlook knocks stocks back despite german <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: eight members of an <unk> <unk> group have been arrested in connection with a string of attacks on france 's rail network , french interior minister <unk> <unk> announced <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: train sabotage : eight leftwing anarchists arrested : minister </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: angola cup final result on wednesday : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : angola cup final <unk> # UNK # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> foreign minister <unk> <unk> on wednesday repeated <unk> 's <unk> of a deadly us raid on a syrian village that he said harmed relations between the neighbouring countries . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: iraq again denounces us attack on syrian village </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new zealand share prices closed <unk> percent lower wednesday after investors took their lead from further weakness in overseas markets , dealers said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new zealand shares close down <unk> percent </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> moved into second place in italy 's serie a with a <unk> win over <unk> at <unk> 's <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : <unk> up to second in impressive style </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: us shares managed modest gains in morning trade thursday as investors looked for bargains after a punishing three-day market selloff and shook off more bleak economic and corporate news . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: wall street struggles higher after three-day rout </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: china plans to punish singers who UNK for `` cheating the public , '' the ministry of culture said thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> features </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> hit a fighting century but could n't stop <unk> winning the second day-night international by <unk> runs here on friday , securing an <unk> <unk> lead . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : <unk> 's hundred in vain as <unk> clinch series </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> coach manuel <unk> was <unk> disappointed at dropping two points in the title race as <unk> scored a <unk> minute equaliser to snatch a <unk> draw on sunday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : <unk> <unk> dropped <unk> points </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: stock markets in the gulf states plunged on the week 's opener sunday as panic from the fallout of the global economic crisis continued to dampen investor sentiment . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: gulf stocks plunge as <unk> fails to halt panic </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: international mediators raised concerns monday over fresh fighting near <unk> 's disputed <unk> <unk> region that left at least one soldier dead at the weekend . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mediators raise concerns over deadly <unk> fighting </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hosts <unk> defeated england by <unk> runs on monday to take a <unk> lead in the <unk> one-day series . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> UNK advisory </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: leading <unk> exiles began a week-long meeting monday in northern <unk> that could usher in a more radical approach to their long struggle against chinese rule in <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> plot future after <unk> lama admits failure </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: barclays , the second-biggest british bank , said <unk> that investors from the oil-rich gulf had agreed to amend terms of their proposed capital injection worth billions of dollars . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: barclays proposed gulf investors amend injection plan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chinese share prices plummeted <unk> percent on <unk> as <unk> concerns over china 's slowing economy and weakness in regional markets triggered heavy selling , dealers said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chinese shares close down <unk> pct </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: david beckham 's career should be over , england 's <unk> world cup winner martin <unk> said following the la galaxy midfielder 's run of substitute appearances . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : beckham career should be over says world cup winner <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> number two <unk> <unk> warned us president-elect barack obama against sending more troops to afghanistan , in an internet UNK released on wednesday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> warns obama against sending troops to afghanistan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: there could be a nasty surprise under the tree for the us economy this christmas . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: grim <unk> for us christmas shopping season </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: opponents of the leftist government of president <unk> <unk> said wednesday they had enough votes in congress to annul the disputed <unk> # municipal elections in <unk> . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> client advisory on <unk> cricket coverage </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: kevin <unk> scored <unk> points in his return after a <unk> suspension and the boston <unk> ripped detroit <unk> here thursday in a rematch of last season 's nba semi-finals . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: basketball : <unk> makes triumphant return as <unk> top pistons </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> share prices opened slightly higher on monday following a big rally on wall street ahead of the weekend . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> shares open up <unk> percent </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> striker <unk> <unk> could be ruled out for several weeks if a specialist decides the england international needs surgery to cure a calf injury . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : <unk> to see specialist over calf injury </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: oil prices roared higher towards <unk> dollars on monday as equity markets surged on government action aimed at tackling a severe economic downturn . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: oil prices soar towards <unk> dollars </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's central bank on monday cut its interest rate by <unk> basis points to <unk> percent in a bid to boost growth amid the global economic crisis . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> central bank cuts key interest rate to <unk> pct </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: french champions <unk> knocked <unk> out of the champions UNK with a <unk> victory here on <unk> as they ensured their own place in the <unk> stages . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : <unk> sent tumbling by <unk> defeat </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the world 's biggest miner bhp billiton announced <unk> it was dropping its controversial hostile takeover bid for rival rio tinto due to the state of the global economy . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bhp billiton drops rio tinto takeover bid </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new <unk> violence left five policemen and <unk> militants dead in afghanistan , authorities said wednesday , as a un security council team continued a tour to assess <unk> development . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> killed in afghanistan as un security council team tours </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's powerful army chief wednesday told anti-government protesters they must leave several key sites that they are occupying , including <unk> 's international airport . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: thai protesters must leave airport other sites : army chief </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: two units of <unk> 's industrial conglomerate <unk> plastic group have agreed to lend <unk> million us dollars to <unk> technology of the united states , the companies said wednesday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's <unk> to lend <unk> <unk> mln dollars for acquisition </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: writer juan <unk> was thursday awarded the <unk> prize , the top literary prize in the <unk> world , culture minister <unk> antonio <unk> announced . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: spanish novelist juan UNK wins top literary prize </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chinese shares were up <unk> percent by midday thursday after the central bank announced its biggest interest rate cut in a decade to boost weakening economic growth , dealers said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chinese shares up <unk> pct at midday </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> midfielder paul <unk> launched a passionate defense of <unk> football on friday after another week of european misery . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : <unk> football is not a joke says <unk> star </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new zealand shares closed <unk> percent higher friday in their fourth straight rise as investors welcomed an easing of volatility in world markets , dealers said . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new zealand shares close <unk> percent higher </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new zealand completed their second grand slam in three years and third overall with a <unk> win over england at <unk> here saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> : all blacks down england to seal grand slam </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the dollar was quoted at <unk> yen here early <unk> , up from <unk> yen monday afternoon and from <unk> yen in new york late monday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: dollar at <unk> yen in early <unk> trading </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: israel and <unk> signed here on <unk> an agreement to open trade representation offices in each other 's country . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> israel sign accord to open trade offices </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a senior us official arrived here wednesday for a week of talks with chinese counterparts aimed at settling a long-running dispute over copyright piracy in china . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us official in <unk> for talks on copyright piracy row </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a court wednesday ordered a full investigation into the abductions of <unk> people amid widespread official persecution of leftists during argentina 's <unk> military government . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: court orders investigation of <unk> abductions in argentina </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: torture and <unk> of prisoners , suspected terrorists and even children is increasingly widespread in many countries , according to the latest un report on torture released thursday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: torture is common practice in many countries : un report </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: grid positions after the final qualifying session in the <unk> motorcycle grand prix at the <unk> circuit , west java , saturday : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> motorcycle grand prix grid positions </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: mystery fumes swept through a <unk> subway station sunday forcing the <unk> of one woman and causing a new scare over the aum supreme truth cult . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: woman injured in <unk> subway fumes scare </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: foreign ministers and officials of <unk> islamic countries met in the <unk> capital wednesday to discuss the contribution of the islamic world to the reconstruction of this <unk> country . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: islamic countries discuss aid for <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: boat dealers and builders participating in the first-ever international boat show in china are forecasting a boom in leisure - craft sales within the next five years . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: boat builders see boom in UNK sales UNK picture by <unk> <unk> ying </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: opec 's president ammar UNK arrived late friday in <unk> on the third stage of a tour of the oil states of the gulf . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: opec president arrives in <unk> on next stage of gulf tour </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: scoreboard of the <unk> cup one-day international between <unk> and <unk> here on monday : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> v <unk> scoreboard </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russian forces in <unk> began a limited withdrawal monday , but warplanes kept bombing villages in the southeast of the <unk> mountain republic despite a unilateral ceasefire promise by <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> start troop withdrawal but continue bombing in <unk> by <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> prime minister <unk> <unk> accused <unk> on <unk> of supporting the israeli assault on <unk> guerrillas in <unk> as he announced plans to visit london . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> to visit <unk> which he accuses of backing israel </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the pilots of an american airlines jetliner that crashed into a <unk> mountain in december killing all <unk> people aboard were tired and confused , according to a preliminary investigation . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: human error blamed for fatal american airlines crash </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: secretary of state warren <unk> widened consultations on an <unk> ceasefire wednesday by including egypt and <unk> arabia in the effort , an official said . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> widens consultations over <unk> crisis </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: german parliament called on the international olympic committee on thursday to do more for women in sport . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> told to help women </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's <unk> <unk> completed his hat-trick of men 's singles badminton titles at the european championships , winning the final here on saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> wins singles title </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: four people were killed and more than <unk> injured when a passenger train derailed near here sunday morning . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: four killed in train accident </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: china and russia are to sign bilateral treaties on police cooperation during russian president <unk> <unk> 's visit here this week to help combat <unk> crime , <unk> reported <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: china russia to sign police treaties during <unk> visit </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president bill clinton announced reforms of the central intelligence agency aimed at restoring credibility in an espionage agency tarnished by the discovery of a russian <unk> in its midst . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: clinton announces us intelligence reforms </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: french foreign minister <unk> de <unk> arrived thursday in <unk> from damascus on a fresh round of shuttle diplomacy and immediately went into talks with <unk> leaders . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: de <unk> in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hong kong shares closed down <unk> percent on friday due to an absence of buyers and fresh incentives , dealers said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hong kong shares end <unk> percent lower </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> 's <unk> hit a <unk> minute winner at <unk> cup finalists <unk> <unk> , enough to knock the <unk> off top spot in the <unk> on saturday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK knocks <unk> off top spot </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> police say a bomb explosion that killed an ethnic <unk> child in the tense province of <unk> was an accident , the <unk> news agency said sunday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> police in <unk> say bomb that killed child was accident UNK new series </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: gold opened lower here on monday at <unk> <unk> us dollars an ounce , against friday 's closing rate of <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: gold opens lower in hong kong </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: israeli prime minister <unk> peres said monday he was confident the ceasefire in <unk> would hold because it was in the best interests of both countries as well as syria . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: peres confident ceasefire will hold </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a chinese pro-democracy activist , who spent last year in self-imposed exile in france , has been arrested by police in <unk> , a report said <unk> . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chinese dissidents arrested in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> boxer <unk> UNK has died after sustaining serious head injuries during a loss to former thai champion <unk> <unk> here , hospital officials said sunday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: boxing : <unk> fighter dies after <unk> bout </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: tenth seed <unk> <unk> defeated <unk> <unk> <unk> , <unk> , <unk> here sunday in the final of the <unk> million dollar <unk> <unk> masters series event . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tennis : <unk> 's <unk> UNK <unk> men 's title </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> share prices fell <unk> percent monday after investor sentiment was rattled by speculation the central bank will raise interest rates this week , dealers said . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> shares down sharply on interest rate fears </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: global steel giant UNK on monday announced a share buyback program to repurchase up to <unk> million dollars <unk> <unk> million euros <unk> worth of shares . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> announces share buyback program </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: japan on <unk> said it was taking a `` cautious position '' on a civilian nuclear agreement between <unk> and the united states . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK cautious on <unk> us nuclear deal </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's senior pro <unk> <unk> on wednesday broke his silence on the world cup debacle , saying the first round ouster had `` shattered the team 's dream . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : <unk> shattered by world cup exit </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: israeli soldiers entered the gaza strip and clashed with militants on wednesday , the first major incursion into the territory since a <unk> ceasefire , <unk> security sources said . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK israel troops launch incursion into gaza </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russian authorities want a european body to investigate the deaths of <unk> seals in the <unk> sea , environmental officials said wednesday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russia wants investigation into seal deaths </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: mark <unk> , the <unk> masters winner , doomed his chances of capturing this year 's first major championship if tradition holds by winning the masters <unk> contest here wednesday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: golf : <unk> hopes to break masters <unk> jinx </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: defending national basketball association champions <unk> , who have battled injuries all season , booked their playoff berth thursday with a thrilling overtime victory over <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: basketball : nba champions heat book return trip to playoffs </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> defender <unk> is out for the rest of the season with a knee injury , his porto coach <unk> <unk> said saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : <unk> out for season </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a high-level us delegation led by former un ambassador bill <unk> arrived in north korea on sunday , official north and south korean media said . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us delegation arrives in north korea : report UNK UNK north korea 's media confirms arrival details </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the new york yankees placed slugger <unk> <unk> on the <unk> disabled list on sunday with a strained left hamstring . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: baseball : <unk> back on disabled with strained hamstring </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: key events since the overthrow of <unk> leader <unk> hussein four years ago : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> advisory </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: us oil services giant <unk> said monday it had wrapped up its work commitments in iran and was no longer conducting any projects in the islamic republic . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> winds up iran work </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: world oil prices rebounded on <unk> after plunging a day earlier on profit - taking , as key crude producer iran again tested western nerves , analysts said . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: oil prices rally after easter slump UNK UNK with closing prices changes UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: five candidates contesting east <unk> 's presidential election filed a formal protest wednesday saying the poll was not fairly conducted , and that counting should be stopped immediately . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> candidates file formal protest on poll UNK UNK UNK details </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: christian conservatives -- <unk> in the last two us presidential elections -- may have less success in getting their pick elected in <unk> , political observers say . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: christian conservatives power diminished ahead of <unk> vote by <unk> <unk> UNK UNK UNK detail </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the un security council on thursday strongly condemned the suicide car bombings in <unk> and urged all states to help <unk> bring the perpetrators and sponsors to justice . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tennis : houston <unk> results <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: who 's saying what at the <unk> cricket world cup : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : world cup UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: two lines of <unk> riot police linked arms to trap protesters on a street in central <unk> on saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> world news summary </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: france 's <unk> <unk> claimed his <unk> career <unk> pole position here saturday , leading qualifiers for sunday 's long beach <unk> grand prix . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: auto : <unk> grabs <unk> pole </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results here on sunday in the paris marathon : - UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: marathon de paris les UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: wayne <unk> admits manchester united 's breathtaking form has <unk> him for the decisive final weeks of the season . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> world news service no unauthorized <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: nearly <unk> people <unk> to the netherlands last year , marking an increase for the fourth year , statistics showed on monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: immigration to netherlands increased in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> man was sentenced to <unk> months in prison by a los <unk> court on monday on charges on trafficking exotic <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us jails UNK man for butterfly smuggling </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: south africa rode on andrew hall 's maiden <unk> burst to storm into the world cup semi-finals with an emphatic <unk> win which humiliated england here on <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : hall and smith fire south africa into semi-finals by UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the death toll from last month 's blast in <unk> climbed to <unk> as two more bodies were discovered in the wreckage , the mine 's owner said wednesday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russian mine toll rises to <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> elders accused <unk> troops on wednesday of breaking a truce by attacking islamist insurgents in southern <unk> , sparking clashes that killed at least seven civilians . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> elders blame <unk> troops for clashes by UNK UNK UNK UNK UNK throughout </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: mark <unk> pitched the <unk> <unk> in chicago white sox history on wednesday , as the white sox defeated the texas rangers <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: baseball : white sox <unk> UNK UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: riot police broke up scuffles thursday as a key by-election campaign began in <unk> that could bolster former deputy prime minister anwar <unk> 's return to formal politics . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: scuffles as key <unk> by-election kicks off </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: west indies captain brian lara said he will quit international cricket on saturday , ending speculation over his future after his team 's dismal world cup campaign . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : lara quits international cricket UNK UNK details UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a gunman killed a male hostage and himself at nasa 's <unk> space center friday , just days after <unk> people died in the bloodiest school shooting in us history . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> sports schedule for saturday UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: result of an african champions league third round , <unk> match here on saturday : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: african champions league result : young <unk> # <unk> # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: thousands of <unk> chanting <unk> slogans on sunday attended a rally to welcome back a hardline separatist leader who underwent cancer treatment in <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: thousands attend rally for <unk> hardliner </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ireland have been rewarded for their memorable world cup debut by being confirmed as the <unk> best team in the world . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : ireland confirmed in world top ten </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president <unk> <unk> will begin a two-day visit to russia , his country 's main energy partner , on monday for trade talks , the <unk> press office said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> president to visit russia for energy talks </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> man was beheaded by the sword in the southern city of UNK on <unk> after he was convicted of murdering a compatriot , the interior ministry said . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> executed for murdering compatriot </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: us aviation giant <unk> co. said wednesday first-quarter profits lifted <unk> percent from a year ago to <unk> million dollars as new aircraft orders took off . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> profit up <unk> percent soars past UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> seriously damaged the car of <unk> 's ambassador to greece overnight , police said thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: car of <unk> envoy to greece <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an israeli patrol crossed into southern <unk> on thursday and was forced to end its `` violation '' after the intervention of un peacekeepers , the <unk> army said . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: israeli patrol crosses into <unk> : <unk> army </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hollywood <unk> and paparazzi favorite <unk> <unk> lobbied in washington thursday for the world 's orphans and children at risk . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> seeks more aid for orphans poor children </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results and standings after the national league 's thursday baseball games : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: results and standings in baseball 's national league </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: european union foreign policy chief <unk> <unk> called on friday for the united states to open a `` channel of communication '' with iran on all subjects . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : english cricket scores </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: sri lanka were <unk> after <unk> overs in the world cup final against australia at <unk> oval on saturday chasing <unk> to win . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ice hockey : <unk> playoff results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: environmental groups monday called for immediate and decisive action on climate change , amid concern that powerful nations will seek to water down a <unk> aimed at tackling global warming . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: green groups urge action now on climate change </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: sri lanka 's <unk> tiger rebels said monday that they hit a military war plane on a bombing mission over their territory with anti-aircraft fire . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tennis : <unk> <unk> and <unk> open results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> share prices fell <unk> percent in morning trade <unk> as losses monday on wall street overnight kept buyers at bay , dealers said . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK shares down <unk> percent by lunch </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an explosion in iraq 's restive northeastern province of <unk> killed two us soldiers and wounded two more , the military reported monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: two us soldiers killed in iraq blast december toll <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: vietnam airlines on wednesday reported UNK profits for <unk> despite increased flights , passenger numbers and turnover , a shortfall it blamed in part on higher fuel costs . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: vietnam airlines profit below target </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president <unk> <unk> will spend orthodox christmas this weekend with the country 's troops serving in iraq , his cabinet said thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> president to visit troops in iraq </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: us stocks put on a late closing <unk> thursday as oil prices tumbled on <unk> winter temperatures , despite mixed sales reports from large retailers following the christmas holiday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us stocks gain as oil prices slump UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: six-party negotiations aimed at convincing north korea to abandon its nuclear weapons are expected to resume later this month in <unk> , the us state department said friday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us expects north korea nuclear talks to resume this month </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the us embassy in <unk> on saturday confirmed the kidnapping of an american , saying he was working in iraq as a private security contractor . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us embassy confirms UNK of us citizen in iraq </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russian world <unk> # <unk> <unk> became the fifth withdrawal through injury or illness at the <unk> international wednesday , retiring from his second round match with a foot injury . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tennis : <unk> pulls out of <unk> with injury </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the european commission unveiled sweeping plans wednesday to diversify eu energy sources , slash carbon emissions by <unk> percent and enforce fuel competition . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK eu unveils vast energy plan to protect supplies environment </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> midfielder arnold m ` UNK has been loaned till the end of the season to english premiership club <unk> , the french first division club said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : m UNK joins <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> electronics maker sharp corp. announced plans friday to ramp up its output of flat screen televisions with new production lines in japan and <unk> in response to <unk> demand . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: sharp to expand flat tv production in UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a woman street cleaner and her three young daughters were killed saturday when a bomb in a metal container exploded in <unk> , police said . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mother three daughters die in in <unk> blast </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: los <unk> galaxy coach frank <unk> is not sure when david beckham is coming , but he knows where the former english captain is going . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : galaxy coach wants beckham in central midfield </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: croatia 's fourth seed ivan <unk> became the <unk> player eliminated on the opening day of the <unk> open here monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tennis : fourth seed <unk> knocked out of <unk> open UNK UNK details </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: italian police have uncovered a scheme by which the families of some <unk> illegal immigrant minors were <unk> into paying a gang for their freedom , police said monday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: six arrested in <unk> for trafficking immigrant minors </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an international terror suspect who had been under a controversial loose form of house arrest is on the run , british home secretary john reid said <unk> . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: international terror suspect slips net in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a third vessel may have contributed to a fatal collision between a container ship and a passenger <unk> in italy 's busy <unk> strait , press reports said wednesday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: third vessel implicated in <unk> strait collision </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK beware : spouses caught cheating in michigan could end up spending the rest of their life in prison . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cheat on your spouse in michigan and spend life in prison </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president <unk> <unk> and high-ranking syrian <unk> party officials met here thursday and affirmed their joint aim to improve ties between the two countries , official media said . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> president syrian <unk> party strive for better ties </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> us troops killed in helicopter crash near <unk> UNK UNK with us toll UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: insurgents killed five us troops across iraq 's sunni province of <unk> , bringing us losses in a single day to <unk> , the military reported sunday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: five us soldiers killed in western iraq </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: china reinsurance , the nation 's only reinsurer , received a <unk> government fund injection in preparation for listings at home and abroad , state media said monday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chinese reinsurer gets # billion dollar injection ahead of <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> said monday vehicles carrying hazardous material will be fitted with an UNK in case of extremist <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK to install UNK on potential truck bombs </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: for a tiny caribbean nation , cuba has a big reputation for its sporting <unk> but that is increasingly coming under threat . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: boxing : cuba worried as talent continues to desert </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a german registered container ship ran aground at the entrance to the french port of le <unk> early <unk> , but authorities said there were no casualties . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: container ship runs aground in french port </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the european commission on <unk> called on germany , france , italy and <unk> to step up efforts to improve their public finances despite booming tax revenues . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eu urges germany france italy and <unk> to improve finances UNK UNK details </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> presidency battle between <unk> <unk> and incumbent <unk> <unk> was <unk> up on <unk> with heavyweight german football legend <unk> <unk> throwing his weight behind <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : <unk> backs <unk> in <unk> fight UNK UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> security forces detained an armed hijacker who forced a <unk> plane heading to the troubled <unk> region to to n ` <unk> on wednesday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> troops seize hijacker of <unk> plane UNK UNK throughout with detail on hijacker </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: germany 's <unk> <unk> and robin <unk> won their first major title when they claimed pairs gold at european figure skating championships here on wednesday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: figure skating : <unk> claim european pairs gold </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results from day <unk> of the <unk> open at <unk> park here thursday <unk> x <unk> <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> open results day <unk> <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: four children were killed and another three wounded thursday when an old mortar <unk> exploded as they played with it in afghanistan 's capital , police said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: children killed by old explosives in afghanistan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> core consumer prices rose by <unk> percent in december from a year earlier , the government said friday as <unk> pressure eases in the country . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK UNK 's core consumer prices up <unk> percent in december </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: vaccinations have reached record levels in poor countries , saving the lives of some <unk> million children , a global <unk> program funded by computer billionaire bill gates said friday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: vaccines work <unk> in poor countries : gates </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: sweden 's jens <unk> won his second men 's world cup slalom within <unk> hours here on sunday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket : south africa v <unk> scoreboard </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: insurgents attacked a police post in a taliban heartland in southern afghanistan overnight , killing a policeman and wounding two others , officials said monday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK kill policeman in southern afghanistan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: rio <unk> admits manchester united 's recent defensive problems are causing concern ahead of the crucial , closing stages of the season . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: football : united UNK a worry for rio by ian UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russia is cautiously optimistic about upcoming six-nation talks aimed at dismantling north korea 's nuclear weapons program , deputy foreign minister alexander <unk> was quoted as saying <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cautious optimism for <unk> nuclear talks : russia </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: some <unk> countries met here wednesday to set national quotas for tuna whose total catch has been reduced to prevent the <unk> popular fish being hunted to extinction . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: nations to set tuna quotas amid extinction fears </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: canada and the united states have launched a satellite mapping project of north america to better monitor biodiversity and climate change in the region , natural resources canada said wednesday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: canada us launch satellite UNK of north america </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chelsea owner roman <unk> and england captain david beckham top english soccer 's rich lists . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: second test scoreboard </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: tom UNK , a pop artist best known for his modern take on the <unk> female nude , has died at age <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: pop artist tom UNK is dead at <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: jack <unk> , a <unk> reporter and newspaper columnist who wrote books on robert f. <unk> and boxing <unk> don king , has died . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: u.s. columnist author jack <unk> dies at <unk> ; wrote books on bobby <unk> don king </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: three city police officers have been arrested on suspicion of shooting two arizona tourists they were allegedly trying to rob , federal police reported on wednesday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mexican federal agents arrest city police accused of robbing arizona residents </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in a land of magic and <unk> , beyond the waves of the arabian sea , lives a hero whose soul will forever remain american . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> of globalization _ <unk> has its own <unk> tailored to local tastes </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the vatican 's official <unk> translation of the italian text of pope john paul ii 's <unk> during midnight mass in st. peter 's <unk> : UNK </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: text of pope 's midnight mass <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it 's the perfect <unk> morning at <unk> <unk> 's <unk> -- warm and bright , with luxury <unk> powering through the sparkling blue atlantic ocean some <unk> stories below . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a superstar real estate agent plots his comeback </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a european union naval spokesman says pirates have freed a <unk> <unk> they held for more than seven months . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> pirates free <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results monday from the la women 's tennis championships , a $ <unk> <unk> tour event played on <unk> at the home depot center <unk> <unk> in <unk> <unk> : UNK </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> tour la women s tennis championships results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> court convicted an outspoken opposition legislator on <unk> of <unk> prime minister hun sen , despite complaints from rights groups that the lawsuit was aimed at <unk> critics . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: court rules politician <unk> <unk> pm </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> security council voted unanimously <unk> to name and shame countries and insurgents groups engaged in conflicts that lead to children being killed , <unk> and raped . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: un to name those that kill children in war </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's air force says one of its fighter jets has crashed on a training flight , killing the pilot . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> air force fighter jet crashes pilot dies </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a suspected u.s. missile strike killed a wife of <unk> taliban chief <unk> <unk> at his <unk> 's house wednesday , <unk> intelligence and military officials said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: officials : taliban chief s wife killed by missile </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: reinsurer <unk> re ag said thursday that its second-quarter net profit doubled as demand increased for its products , particularly life and health reinsurance . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> re s <unk> profit doubles </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> starts its new season in unfamiliar <unk> against very familiar opponents when it takes on west <unk> <unk> in the league championship on saturday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> opens <unk> season at west <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a woman who let her <unk> son drive while she was passed out drunk in the passenger seat has been sentenced to <unk> days in jail . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us mom sentenced for letting son # drive </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the parallels between iran and north korea would seem to <unk> off the same UNK script : start a nuclear program , test some long-range missiles , demand international respect . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> iran use similar script to get their way </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> activist called friday for a <unk> investigation of recent violence in western china , as <unk> people demonstrated outside the chinese consulate in <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> activist calls for probe on unrest in china </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: sonia <unk> was sworn in saturday as the supreme court 's first hispanic justice and only third female member in the top u.s. court 's <unk> history . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> sworn in to top us court </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chinese ace lin dan is poised to become the first player in world badminton championship history to win three successive men 's singles titles . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: lin dan <unk> hat-trick of world titles </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president barack obama 's campaign for a health care overhaul is an intense installment in a long-running story , dating to <unk> <unk> in <unk> . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: analysis : health care debate a long-running story </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president barack obama is presenting the highest u.s. civilian honor to <unk> actors , athletes , activists , scientists and <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: activists actors athletics honored by obama </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results thursday at the <unk> cup , a $ # million <unk> tour event on outdoor UNK at UNK stadium <unk> <unk> in <unk> <unk> : UNK </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK cup results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: israel appears to be quietly halting new building projects in the west bank even as it publicly refuses u.s. calls for an official settlement freeze . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: israeli officials : new west bank projects frozen </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> richards shook off years of disappointment with her first major title in the <unk> meters , pumping her fist after crossing the line at the world championships . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: richards wins bolt through to semifinals in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: canada 's prime minister has <unk> on seal meat in a gesture of support for the sealing industry . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: canadian pm has seal meat </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: greece international central defender <unk> <unk> says he intends to join <unk> after receiving an offer from the premier league club . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> says intends to join <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ivory coast striker <unk> <unk> is set to leave <unk> <unk> for french first division side <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> set to sign for <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: don hewitt , the <unk> <unk> who invented the highly popular tv <unk> `` <unk> minutes '' and produced it for <unk> years , died wednesday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> news pioneer don hewitt dies at <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: scoreboard at stumps thursday on the third day of the first cricket test between sri lanka and new zealand at <unk> international stadium : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: sri lanka vs. new zealand scoreboard </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: south african teenager caster <unk> received her gold medal thursday for her <unk> win at the world championships . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> receives medal for <unk> win </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an <unk> police official says a bomb attached to a small truck has exploded near a vegetable market in southern <unk> , killing two and wounding <unk> . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: iraq : # killed in <unk> truck bombing </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: u.s. home <unk> posted the largest monthly increase in at least <unk> years last month as first-time buyers rushed to take advantage of a tax credit that expires this fall . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> home sales surge more than # percent </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> us coast warned as bill stays offshore </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> ap <unk> -- results from the <unk> first division <unk> home teams listed first <unk> : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> football results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: diego <unk> predicted a victory against <unk> in their world cup qualifier at <unk> on <unk> # . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> predicts victory over <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results <unk> from the pilot pen , a $ <unk> <unk> and $ <unk> <unk> event played on <unk> at the <unk> tennis center <unk> <unk> in <unk> <unk> : UNK </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> pilot pen results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: police say gunmen have opened fire on a group of men in a street in the southwestern <unk> city of <unk> , killing three . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: # killed in shooting in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results wednesday from the pilot pen , a $ <unk> <unk> and $ <unk> <unk> event played on <unk> at the <unk> tennis center <unk> <unk> in <unk> <unk> : UNK </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> pilot pen results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> royal palace says a doctor has ordered king <unk> vi placed in <unk> for five days for a <unk> infection . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> s king <unk> with <unk> virus </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it 's not green cheese , but it might as well be . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: moon rock in <unk> museum is just <unk> wood </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: facebook agreed thursday to give users more control over the information they share with third-party applications like games and <unk> in response to concerns raised by canadian privacy officials . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: facebook agrees with canada on privacy controls </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: manchester united will begin the defense of the league cup against premier league newcomer <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: man united opens league cup defense against UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> from the first round of the spanish <unk> football league <unk> home teams listed first <unk> : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: spanish football <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: formula one leader <unk> button crashed out of the <unk> grand prix on sunday and will finish out of the points for the first time this season . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> leader button crashes out of <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: german media company <unk> said monday it lost money in the first half of the year on declines in advertising revenue and consumer spending . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> makes <unk> net loss of <unk> million </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: french judges on monday ordered UNK generale trader <unk> <unk> to stand trial over transactions that cost the bank billions of euros <unk> dollars <unk> , a judicial official said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: french trader to stand trial over <unk> losses </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: waving banners against immigration , european integration and president <unk> <unk> , about <unk> far-right supporters marched wednesday while unions were divided on the traditional labor day . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: big national front march on labor day while unions divided eds : UNK with union march drawing up to <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: three u.s. warships with some <unk> marines on board moved toward <unk> 's shores thursday after fierce fighting erupted on a strategic bridge in the capital . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chief <unk> warlord calls for end to cease-fire eds : UNK with u.s. efforts to broker a cease-fire </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a man who pushed his wife to her death from a <unk> balcony has been executed , the china women 's news reported friday . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: man who pushed wife off balcony others executed in china eds : UNK with reports of # other executions </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: for the first time since this caribbean nation gained independence in <unk> , a woman is running the country - if only temporarily . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: for first time woman acting as prime minister </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> snapped thomas muster 's <unk> clay streak saturday at the <unk> open , posting a surprisingly one-sided win over the <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eds : contains items on <unk> <unk> jordan <unk> arabia </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> authorities have reopened the roads leading from the city of <unk> to the <unk> self-rule area in northern iraq , the official <unk> news agency has reported . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> authorities reopen roads to <unk> area </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the speaker of germany 's parliament urged greater participation of women in technical fields , at monday 's start of an international forum on women 's role in leadership . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: forum hears UNK for more women in technical fields UNK photo UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: home-grown champions have been few and far between at the italian open . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: italy 's woes continue with five losses on first day eds : matches begin at <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the governing party on <unk> suspended a veteran local legislator who was expelled from the senate because of ethics violations . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: governing party suspends senator after expulsion from senate by <unk> <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: for five years , the supreme patriarch of <unk> <unk> has been marching for peace in a vain effort to save thousands of young men from marching to war . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK mixed ; gold down </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a malaria epidemic in <unk> is in danger of spreading into neighboring countries , the world health organization said thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: who fears malaria outbreak in <unk> could spread </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> shipyard workers rejected a wage offer friday and said they would strike next week , shutting down key shipyards and offshore oil construction yards . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: sultan <unk> shah cup at a glance </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russian border guards in <unk> seized <unk> kilograms <unk> <unk> pounds <unk> of raw opium bound for smuggling abroad , the guards ' command said saturday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russian border guards seize <unk> pounds of raw opium </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the shape of <unk> 's future government could be decided sunday as politicians huddled into meetings to work out alliances between parties and <unk> together a coalition . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: parties meet to work out a coalition to rule <unk> by UNK roy </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: john <unk> , who lost his job as general manager of the <unk> <unk> last week , was fired as coach of the national basketball association team on monday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> fire <unk> as coach two assistants </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: about <unk> <unk> army soldiers flew to turkey on <unk> , where they will receive three months of training on new weapons the <unk> government expects to receive . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> soldiers leave for training in turkey with <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in <unk> <unk> <unk> was president of the jury at <unk> , the world 's largest film festival . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK guide UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: if the communists win presidential elections in <unk> their economic program includes plans to <unk> private fortunes and property and stop <unk> from going abroad , a newspaper reported wednesday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: report : communist program would <unk> money and property </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> insurer <unk> <unk> said wednesday its first-quarter net profit rose <unk> percent and turnover rose <unk> percent , largely due to better results in life and general insurance . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> insurer <unk> reports net profit rise </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russia will ease its currency gradually lower against the dollar during the second half of <unk> but at a rate lower than russian inflation , top officials announced thursday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> olympic qualifying at a glance </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> leader <unk> <unk> will visit greece next week , the government said thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> to visit greece </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a delighted president nelson <unk> met one of his screen <unk> friday , american actor <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: star struck <unk> meets american screen legend UNK photos available </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> `` guitar '' watson , one of rhythm and blues ' most influential <unk> , has died of a heart attack after collapsing on stage in japan . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: rhythm and blues musician <unk> guitar watson dies in UNK eds : UNK UNK UNK UNK : rob <unk> ... to UNK with </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> of france in a UNK captured a wet and wild <unk> grand prix sunday after pole winner and two-time defending champion michael <unk> crashed on the first lap . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: formula one standings </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russian troops backed by aircraft monday bombarded a village held by <unk> rebels as insurgents stepped up attacks on government posts in the capital , officials said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: fighting escalates in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: marc <unk> upset <unk> becker in straight sets to pace <unk> over germany in the opening round of the world team cup monday , while spain beat sweden <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> spain win opening matches </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: this time , there were no <unk> conditions facing colin <unk> at the <unk> golf club . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> bad day advances to semifinals eds : UNK previous </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a small arab party headed by an adviser to <unk> <unk> on <unk> quit the race for israel 's parliament after polls suggested it would fail to win a seat . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's adviser withdraws candidacy in israeli elections eds : UNK with peres comment in <unk> graf </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: two cabinet members submitted their resignations to former premier tansu <unk> on wednesday as part of her attempt to tighten the grip on premier <unk> <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> increases pressure for confidence vote eds : leads with cabinet ministers proposing resignations </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> peng soon , a four-time <unk> singles badminton champion and <unk> 's most famous <unk> , has died of pneumonia . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: badminton ace dies </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: second-seeded <unk> <unk> <unk> disposed of south african amanda <unk> <unk> , <unk> thursday in the third-round of the <unk> open . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> easily advances eds : will be UNK ; stands as <unk> UNK to some lines to </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: waving <unk> and telling jokes , more than <unk> workers took to the streets of the capital friday to protest falling living standards and low salaries . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> workers protest poor living conditions with UNK photo </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a group of <unk> rican businessman will visit chile to explore possible new markets , this u.s. commonwealth 's economic development bank announced . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> rican trade mission to visit chile </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an american delegation scouring the former <unk> union for information on u.s. soldiers missing in action meets with <unk> government leaders this week , a news agency reported monday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us delegation arrives in <unk> to look for <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK UNK scored twice wednesday as <unk> beat new zealand <unk> in the four nations field hockey tournament . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eds : will be led with later <unk> africa match <unk> beats new zealand </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: high productivity does n't always <unk> with high profits in the auto industry . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: study shows auto plant productivity profits not always linked with auto UNK auto UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: armed with a `` magnificent seven '' of young guns from an experienced <unk> youth squad , portugal could be the surprise of the european championship finals . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: armed with experienced young guns portugal hopes to surprise europe </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a key international bank thursday revised upward to <unk> <unk> trillion the average amount of money handled daily by the world 's currency markets . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK by source until <unk> UNK friday final bis figures : daily global currency trade of <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: german foreign minister <unk> <unk> insisted friday that his government would not abandon the half million ethnic <unk> living in <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: german foreign minister visits <unk> hails reforms eds : UNK throughout with agreement signed <unk> comments on </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: his final words from the top , <unk> by radio from <unk> 's frozen peak , were triumphant , jubilant . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: death raises questions about south african team by <unk> guy </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a strong earthquake shook southeastern turkey on thursday , killing at least <unk> people and collapsing a school dormitory on more than <unk> children . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: strong quake kills at least <unk> in turkey ; scores of children UNK in UNK school dormitory </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: france midfielder <unk> <unk> will miss the confederations cup in <unk> because of minor leg injuries , while patrick <unk> is an almost certain absentee , france 's coach said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: france 's <unk> to miss confederations cup <unk> highly doubtful </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: remarks by president george <unk> bush announcing the end of major combat operations in iraq thursday evening from the deck of the aircraft carrier uss <unk> <unk> : UNK </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: text of president george <unk> bush 's speech aboard the uss <unk> <unk> with <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> is <unk> <unk> on the u.s. coalition 's list of the <unk> most-wanted figures . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> : reported arrest </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: formula one teams have agreed to a complete ban on automatic <unk> and launch control for <unk> , auto racing 's governing body said friday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: fia reaches agreement to keep drivers in control of cars </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: rebels and government forces signed a <unk> cease-fire agreement saturday , even as insurgents alleged fresh attacks by government forces . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ivory coast government rebels sign <unk> cease-fire ; fighting reported </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: muslim guerrillas attacked a southern <unk> town and took hostages sunday in fighting that killed at least four people , the military and rebels said . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> rugby league results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> celebrated its independence day by upsetting russia <unk> , and slovakia remained unbeaten after <unk> <unk> <unk> at the world ice hockey championship on sunday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> upsets russia <unk> slovakia <unk> <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the rope on the <unk> is still broken . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: british reopen their <unk> embassy but they ca n't call it that yet </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it was a <unk> feeling he hoped never to experience again , least of all three months later . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: nasa chief found himself <unk> columbia tragedy when communication was lost during russian UNK 's descent </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> rose will be back in `` gypsy '' thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> misses performances but will return to critically acclaimed gypsy revival </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> blue jackets <unk> forward david <unk> for next season . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: blue jackets sign UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> said thursday he had two one-hour meetings with the new owners of the boston <unk> but no deal has been completed for him to return to the franchise . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> says no deal completed with <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: exxon mobil has launched a <unk> <unk> <unk> oil plant in <unk> in a joint venture with state-owned <unk> oil company , officials said friday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: exxon mobil launches its <unk> plant in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> government will fine the san <unk> international airport , or <unk> , <unk> <unk> for not upgrading airport facilities , a federal official said friday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> fines u.s. company in charge of country 's airports </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> and <unk> signed a five-year agreement on monday to promote bilateral trade , officials said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> sign new trade deal </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> police announced monday they had recovered about <unk> stolen gold and <unk> <unk> artifacts stolen in february from panama city 's <unk> museum . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: panama recovers stolen <unk> artifacts </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: british foreign minister jack straw <unk> <unk> fighter <unk> <unk> <unk> as '' a great fighter for freedom , '' as he began his two-day visit to south africa . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: british foreign minister visits south africa </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the suicide bomb attacks in <unk> arabia were `` a cowardly and <unk> terrorist atrocity , '' prime minister tony blair said wednesday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: two <unk> missing after <unk> suicide blasts </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a russian scientist facing espionage charges asked for a jury trial on wednesday , the <unk> news agency reported . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: report : russian scientist requests a jury trial </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a young man with a <unk> beard is emerging as a suspect in a series of <unk> explosions at shell stations in southern <unk> , police officials said friday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> police prepare sketches of suspected gas station bombers </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: whether it 's a brilliant smile or tears of joy , <unk> <unk> lets you know how he feels . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> knows when to get serious </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the talks , which lasted about four hours , were scheduled to resume at <unk> a.m. local time <unk> <unk> <unk> <unk> sunday , the mediator said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK <unk> : he said </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: leading the <unk> field at the far turn , <unk> <unk> <unk> victory as he prepared peace rules for the final push to the finish line . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> watches potential victory fade in stretch </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK <unk> officials rebels fail to agree on <unk> peace deal </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> is scheduled to begin tests on an aids vaccine to find out if it is safe when given to healthy adults , officials said monday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> to begin trials for aids vaccine </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> will host <unk> for three test cricket matches and five one-day international matches in august , an official said <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cricket minnows <unk> to tour <unk> in august for test one-day series </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the bush administration raised the u.s. terror alert level to orange <unk> amid fears a wave of terrorist attacks overseas will spread to the united states . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK u.s. raises terror alert level to orange amid concern over terror attacks </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: former president nelson <unk> on wednesday asked the english national soccer team to support south africa 's bid to host the <unk> world cup . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> world team cup results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the nfl will look into the awarding of the <unk> championship to the chicago cardinals instead of the UNK <unk> of <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK <unk> stake claim to <unk> nfl title </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: alan <unk> continued his winning ways by capturing album of the year honors at the academy of country music awards . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: alan <unk> <unk> album of the year as las vegas hosts u.s. country music awards </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the ministry of finance on thursday lowered its economic growth estimates for the year , predicting slower growth of the gross national product , and increased unemployment . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> cuts growth UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: army <unk> tommy franks , who planned and commanded the <unk> wars in iraq and afghanistan , has decided to retire , defense officials said thursday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> franks commander of war in iraq to retire officials say </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: french foreign minister <unk> de <unk> will meet with <unk> <unk> on monday at the <unk> leader 's headquarters <unk> , the french foreign ministry announced friday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: french foreign minister <unk> de <unk> to meet <unk> leader <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: he 's leader of the world 's most populous nation , but a mystery to most . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hu 's debut : western leaders get rare chance to size up china 's new leader </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a powerful earthquake rocked northeastern japan on monday , knocking out power , causing a landslide and disrupting road and rail traffic . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: strong earthquake shakes northeastern UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the archbishop of <unk> believes the church of england should change its teaching on homosexuality to accept gay relationships , his <unk> was quoted as saying in a newspaper <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: archbishop of <unk> believes church should accept gay relationships UNK says </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> scoring machine will be missing a cog when it meets <unk> 's cruz <unk> on wednesday in a <unk> <unk> for the <unk> <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: french open show court schedules </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a federal jury ordered ebay inc. to pay $ <unk> million for violating patents filed by an attorney in a ruling that could change how the online auction house operates . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: federal jury orders ebay to pay $ <unk> million in patent dispute case </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: police arrested about three dozen opposition lawmakers from a provincial assembly wednesday during two protests against constitutional changes made by <unk> 's president to increase his power . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> police arrest more opposition lawmakers in second day of protest </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> network plotted terror strikes in australia well before the <unk> <unk> , <unk> attacks in the united states , prime minister john <unk> said thursday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: government says <unk> UNK terror targets in australia before <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: spain 's royal family has a new member _ <unk> <unk> <unk> UNK _ the illegitimate son of former spanish king <unk> <unk> , news reports said thursday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: illegitimate son of former king wins battle to be recognized as part of spanish royal family </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a u.s. congressman expressed concern wednesday over the health of <unk> 's democracy and condemned pending conspiracy charges against a leading government opponent . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: u.s. congressman expresses concern over <unk> democracy under <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: workers at greece 's largest oil refinery threatened thursday to stage a <unk> strike over pay and new labor contracts . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: greek refinery strike threatened over pay labor reforms </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: bureaucrats . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bureaucrats get their share of blame in european constitution crisis </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: at stake for the detroit pistons in game # against the <unk> heat is so much more than their nba playoff lives . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> has to switch u.s. open qualifying sites </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president <unk> <unk> arroyo 's spokesman released a pair of tapes monday as part of a pre-emptive effort to counter expected opposition claims that she fixed last year 's election . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: dispute erupts over UNK purportedly showing arroyo tried to fix election </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> won the first stage at the <unk> <unk> on monday , while six-time tour de france champion lance <unk> finished in the chasing pack . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> wins first stage at <unk> <unk> <unk> finishes in chasing pack </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: long before making `` the <unk> , '' arnold <unk> was a young <unk> <unk> hitting the books at santa <unk> college in the early <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's planned speech causing friction at community college </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: justice ministers from greece and turkey signed an agreement wednesday aimed at assisting reforms ankara must make to join the european union . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: greece to train <unk> judges to help eu accession bid </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: growing up in the slums of <unk> and suffering from a life-threatening heart defect , <unk> UNK UNK feared that she may only have a few more years to live . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: afghan girl with heart defect gets new chance at full life </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: beatings and brutality are common at rio 's youth detention centers , but abuses often go <unk> because there is no outside monitoring , human rights watch said thursday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: human rights watch denounces abuses in youth detention centers in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: share prices were higher on the london stock exchange friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: london share prices close up </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: israeli police have found the olympic gold medal of <unk> gal <unk> that had been stolen , the athlete said saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: israeli police say they have found the olympic gold medal of <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's first two submarines are expected to arrive from france in <unk> , news reports said sunday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: reports : <unk> 's first two submarines to arrive from france in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: michelle wie was rolling her eyes and <unk> her thigh in disgust early in the final round of the <unk> championship . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: wie proves she belongs with <unk> finish </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> truck maker <unk> ab and <unk> oil and gas company <unk> asa are launching a joint venture on fuel cell technology aimed at cutting emission from <unk> engines . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> start joint venture on fuel cell technology to cut emissions </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the european commission on wednesday approved a joint venture between french banks credit <unk> sa and UNK <unk> to combine their securities services businesses . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eu clears french bank credit UNK <unk> joint venture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russia 's constitutional court has agreed to hear the first case challenging president <unk> putin 's abolition of direct elections for governor , a court spokeswoman said thursday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russian high court agrees to hear challenge to putin 's abolition of elections for governor </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the bottom of elena <unk> 's skirt flares with each step , her feet pound and her hands twist as she UNK in a sweltering studio . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> alive and well in u.s. </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: tom <unk> of <unk> will drive for a record seventh victory in the <unk> endurance race at le mans this weekend . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> for record-breaking seventh victory at le mans </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: french open champion <unk> <unk> said she 's better equipped to win <unk> than in years past . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: world cup qualifying : <unk> # <unk> # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: china faces traditional powerhouse germany on <unk> in the round of <unk> at soccer 's world youth championship . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: china success at youth world championship shows preparation for <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new york has brought out its biggest hitter so far in the campaign for the <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> ali to support new york <unk> bid at UNK vote </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: china 's third largest oil producer <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: china 's <unk> offers <unk> <unk> billion for <unk> <unk> chevron 's bid </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: russia 's attempts to enter the world trade organization are on track , the country 's trade minister claimed friday , but trade officials said negotiations still face many obstacles . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russian minister says <unk> membership possible by december but others say obstacles remain </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> criticized `` serious <unk> '' in iran 's elections saturday and urged the new president to address international concerns about the country 's nuclear program . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> criticizes iran election urges move on nuclear concerns </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chris <unk> pitched a <unk> , and <unk> <unk> , jim <unk> and albert <unk> homered saturday night to lead the st. <unk> cardinals over the <unk> pirates <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's <unk> UNK stellar month for cardinals right-hander </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: german chancellor <unk> <unk> arrived in washington for a visit shortened by election-year pressure and overshadowed by the possibility germany will have a new , more <unk> leader this fall . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: election year overshadows <unk> 's shortened visit to washington </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hollywood fashion commentator steven UNK underwent surgery monday for removal of a <unk> kidney that had become infected , a spokesman said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hollywood fashion commentator steven UNK has more surgery </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's national car maker proton expects to export its cars to russia by early next year to boost its overseas sales , a company official said <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> carmaker proton seeks inroads into russia by early next year </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> stocks ticked up wednesday afternoon as the u.s. dollar 's rise to a more than eight month high against the yen helped boost <unk> issues . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK stocks rise as u.s. dollar hits <unk> month high against yen </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: gold <unk> opened thursday at a bid price of <unk> <unk> a <unk> ounce , down from <unk> <unk> late wednesday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: gold opens lower in london </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> co. on thursday named <unk> co. . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> names <unk> chief UNK as new ceo </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: south korea will play for its third straight olympic women 's <unk> gold medal when it meets <unk> saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> south korea advance to final eds : repeating </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> diplomat was among three african men arrested for smuggling <unk> kilograms <unk> <unk> pounds <unk> of marijuana into sweden , the <unk> news agency tt reported friday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> diplomat among three arrested for pot </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK thorpe remembers her difficult marriage to jim thorpe , called the greatest athlete of the modern era , and their harsh life outside the spotlight . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: late <unk> 's wife recalls hard times UNK photo available </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: prime minister <unk> <unk> on sunday said he had issued orders to close the <unk> office of a <unk> legislator who was operating in violation of peace agreements . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> announces closure of <unk> office in <unk> by nicolas b. UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> UNK of germany , on UNK de UNK , won the gold sunday in equestrian show jumping , fulfilling his dream of winning a medal in his first <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: german wins gold in first <unk> ahead of swiss and french </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: legislators said monday they plan to seek more details from <unk> chris <unk> over the sudden resignation of this british colony 's top immigration official . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: legislators seek more details over resignation of official </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> villager attacked his three wives with an ax after they tried to stop him marrying a fourth woman , police said <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK <unk> man attacks three wives with ax </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a wooden warehouse used as a temporary shelter for the homeless went up in flames <unk> , killing <unk> people , three of them children , the fire department said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK to add <unk> ten homeless killed in warehouse fire <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> to public pressure , a subsidiary of the giant <unk> conglomerate said wednesday it will pull out from <unk> u.s. pop star michael <unk> 's scheduled performance here . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: business giant <unk> pulls out of michael <unk> concert </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a bomb exploded in a crowded cafe thursday afternoon and injured seven people , the government said , in the latest of a wave of attacks on city restaurants . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bomb explodes in <unk> cafe injuring seven </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: his leftist fringe party does n't even have <unk> members , but <unk> <unk> UNK is accused of being at the heart of a communist plot to topple the government . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK weekly sports calendar </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: damon hill edged out michael <unk> by only <unk> <unk> of a second , earning the fastest time in friday 's practice for the <unk> grand prix . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hill records fastest time in practice eds : UNK with details UNK ; UNK UNK ; UNK to european </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: echoing a familiar theme in the state-controlled press , a top leader of the military government blamed the <unk> opposition for trying to isolate <unk> and deny it foreign aid . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: top junta leader says opposition trying to <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a newspaper reported monday that an fbi team was en route to israel to question an UNK bomber about last month 's twa jetliner explosion near new york . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: report : fbi team to question bomber in twa explosion </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: david <unk> of the <unk> republic defeated <unk> <unk> <unk> <unk> , <unk> monday in the opening match of the <unk> open . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> UNK ousts veteran UNK in first-round </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a top nato commander inspected a disputed military site <unk> after the <unk> <unk> bowed to pressure from the <unk> peace force . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: officials leave for inspection of <unk> <unk> site eds : UNK with walker UNK after inspection </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the face of french singer <unk> <unk> <unk> from a poster advertisement . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: briefs from canada eds : all dollars are canadian unless otherwise noted </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the government has cut a dramatic hike in electricity charges after workers threatened to strike . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> price hike after union protests </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a south korean novelist who illegally entered north korea was handed over friday to the south korean embassy in <unk> for the trip back home , officials said . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eds : third take UNK at <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: thousands of greek <unk> flocked friday to the funeral of a protester killed two days ago by <unk> troops in the buffer zone running across this <unk> <unk> island nation . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eds : UNK with last match </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> is the fourth generation of his family to work as a <unk> on the <unk> spread around the vast UNK <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> at a glance by the associated press </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: authorities have freed about two thirds of the <unk> people arrested during three days of violent protests over increased food prices , security sources said wednesday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: jordan frees <unk> arrested during food riots UNK photos available </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: top activists arrested after last month 's anti-government rioting are in good condition , a red cross official said saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: arrested activists in good condition says red cross by UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: crew # <unk> # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: philip <unk> europe announced monday it was withdrawing its sponsorship of the <unk> formula one auto racing team three years after the team won its last championship . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: philip <unk> <unk> announce end of <unk> sponsorship eds : UNK with <unk> picking up <unk> ; UNK spelling of </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> share prices bounced back slightly in sluggish trading <unk> after a two-day decline , while the u.s. dollar fell slightly against the yen . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: stocks bounce back after two-day decline dollar mixed eds : UNK up with late dollar closing bond prices comments </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> scored three goals in a <unk> span of the first period <unk> night for a <unk> victory over the <unk> republic in their world cup of hockey opener . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> <unk> republic at world cup </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president alexander <unk> does n't consider himself an <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> outspoken president pushes for more powers eds : UNK # UNK on picket after <unk> graf <unk> UNK to conform ; </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: despite <unk> promises to protect minority rights , <unk> in areas <unk> last year by the <unk> army are frequent targets of assaults and thefts , the <unk> chief reports . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: italian cup at a glance </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a russian plane filled with coal miners slammed into a mountain on a <unk> island near the arctic circle on thursday , apparently killing all <unk> people aboard . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK russian plane crashes in arctic rescue team finds no sign of </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: italy 's <unk> <unk> , coming back from a shoulder injury , saturday reached the final of the <unk> open by <unk> spain 's <unk> <unk> in three sets . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> on rise again reaches final of <unk> open </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a spanish judge opened an investigation monday into the cause of a collision in which five young british tourists were killed a day earlier . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eds : contains items on jordan israel egypt <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: amateur american <unk> <unk> lost concentration and with it a potential upset monday in the <unk> tennis classic . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: france 's <unk> takes first day of competition </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: delegates attending a conference of leading ethnic UNK voted to support the ruling military 's proposed plan to hand power back to civilian leaders , a spokesman said <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: powerful UNK vote to support nigeria 's democracy plan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: rescuers plucked two more bodies from river <unk> on south korea 's south coast wednesday , raising the death toll from weekend flash floods to <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: death toll from floods rises to <unk> ; <unk> still missing </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: both crew members were killed when a civilian helicopter crashed in UNK , some <unk> miles <unk> <unk> kilometers <unk> south of the town of <unk> on wednesday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: two killed in helicopter crash in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: prince charles is setting up a new architectural foundation and hopes to call a truce between modern and traditional styles . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: prince charles sets up modern architectural foundation </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: fast bowler <unk> <unk> gave south africa its only breakthrough in the opening morning of the fifth cricket test at UNK thursday , UNK england opener mike UNK for <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: england <unk> for one at lunch UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: survivors wandered in a <unk> amid burning cars and the twisted steel and concrete of a collapsed building friday after a deadly terrorist bombing outside the u.s. embassy in <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: explosions rock <unk> and stun <unk> with UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: with musical performances and <unk> , russian artists paid tribute monday to <unk> UNK , one of the leading <unk> of this century , who died a week ago . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: russian musicians public pay final respects to <unk> UNK eds : UNK with burial UNK details movie director 's quote </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: share prices on the london stock exchange were sharply lower <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: london share prices lower in afternoon eds : UNK with closing index </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: u.s. diplomats who worked in <unk> buildings were moved into the fortified embassy compound in cairo . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> african embassies tighten security after bombings eds : UNK lead to reflect closures in <unk> <unk> <unk> and </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a senior russian diplomat on thursday accused <unk> of sending its troops and combat aircraft to help the taliban movement crush its opponents in northern afghanistan . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: dollar up gold falls UNK in graf <unk> that gold is down in london ; UNK to conform ; </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: four poles were given sentences from <unk> to <unk> months of prison in connection to one of the biggest <unk> cases in <unk> , officials said friday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: four poles convicted in major UNK case </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: clutching passports and <unk> their watches , <unk> of foreign workers spill into the street outside the <unk> immigration department in a final rush to renew work permits . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: deadline looms over foreign workers amid fear of riots UNK photos UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> <unk> , the UNK of a small egyptian opposition party , died early friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> weather </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the british and irish governments say they will wipe out ira dissidents believed responsible for the <unk> car bombing . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: british irish leaders promise security crackdown on real ira with <unk> ireland UNK victims </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an encounter between army soldiers and separatist guerrillas in <unk> on monday left <unk> militants dead , an official said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ten militants killed in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an israeli businessman turned bank robber whose motorcycle <unk> were legendary in israel during the late <unk> will be released early for good behavior , the prison authority said monday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: israeli bank robbing legend to get early prison release </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new zealand 's ruling coalition <unk> <unk> under pressure from prime minister <unk> <unk> , who warned her minor party counterparts to support her or face an early election . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: coalition minority partner splits as <unk> warns of early </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a controversial play about the assassin of indian independence leader <unk> <unk> has been banned in the country 's most populous state , officials said <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: play on <unk> 's assassin banned in indian province by UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a rash of malaria cases has been recorded in central russia and emergency workers fear an epidemic , a news agency said wednesday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: officials fear spread of malaria in central russia </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: italian police seized <unk> kilograms <unk> <unk> pounds <unk> of hashish being smuggled in containers of <unk> and <unk> <unk> , the ansa news agency reported wednesday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: police seize <unk> being smuggled in containers of UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> security forces on thursday captured two <unk> prisoners , including the killer of an israeli taxi driver , who had escaped the day before , a senior official said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> police UNK two UNK prisoners eds : UNK with more UNK in <unk> graf </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> of italy was the clear winner thursday of the fourth stage of the tour of <unk> while <unk> <unk> <unk> closed the gap on overall leader <unk> <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: results of tour of <unk> cycling race </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: with concrete <unk> <unk> its compound , the u.s. embassy remained on full alert friday against possible terrorist attacks after u.s. missile strikes in <unk> and afghanistan . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: u.s. embassy in <unk> <unk> security after missile strikes by <unk> ung </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: still mourning more than <unk> compatriots slain in the bombing of the american embassy , <unk> on friday expressed mixed feelings about u.s. missile raids on <unk> and afghanistan . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> divided on u.s. raids with UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: with UNK UNK by the associated press </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ethnic <unk> accused <unk> forces of launching a `` strong and massive '' artillery attack sunday on several <unk> villages in southwestern <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> accused of new artillery assault in <unk> by UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the european championship ended with a british triumph and a <unk> loss for wilson <unk> , probably the biggest name of the six-day event . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> loses as <unk> tops the gold medals table eds : UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: greece monday said its security forces were in a state of `` extreme vigilance '' to thwart any retaliatory terrorist attacks against american interests here . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: security forces in state of extreme vigilance </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: retired <unk> alexander lebed urged the russian government monday to appoint a representative to the troubled northern <unk> region , a news agency reported . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: lebed : russia needs new post for troubled <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a bomb exploded on a bus bound for <unk> , <unk> , killing at least <unk> passengers , a private radio station said <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: radio : bomb on UNK bus kills <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> # <unk> <unk> <unk> , <unk> , <unk> UNK , <unk> <unk> , <unk> , <unk> # in <unk> cup second qualifying round , second leg . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> # <unk> # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in chicago , frank thomas and albert <unk> hit consecutive homers for the seventh time this season , and chicago beat <unk> to stop a <unk> losing streak . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> : the sixth </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: where is abu <unk> ? american news reports say the notorious terrorist is detained in egypt . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: by the associated press central league </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> , <unk> # , <unk> , slovakia # <unk> ivan UNK , <unk> <unk> in the european champions cup qualifying game , second leg . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> # <unk> # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> authorities detained <unk> would-be immigrants attempting to cross into greece , paramilitary police in the provincial capital of <unk> said thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> authorities detain <unk> would-be refugees on greek border </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: emboldened by an unprecedented week-long protest by thousands against strongman hun sen , <unk> 's opposition leaders appealed to the nation saturday to drive him from power . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: opposition plans march urge overthrow of hun sen eds : UNK and UNK throughout with opposition urging </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: prime minister john <unk> on sunday called a national election for <unk> # and will campaign mainly on the issue of sweeping tax reform . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: prime minister john <unk> calls national election for <unk> # eds : UNK <unk> UNK in UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: defending champion fc <unk> drew with modest racing de <unk> while <unk> <unk> athletic de <unk> <unk> to take the lead sunday in the spanish first division 's opening round . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> soccer standings </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an earthquake with the magnitude of <unk> shook <unk> 's central region early monday , but there was no injuries or serious damage , seismologists said . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: quake shakes central region </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the chief investigator into the december crash of a <unk> passenger plane can not say <unk> that pilot suicide was not the cause , <unk> government television reported monday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> crash investigator ca n't rule out suicide UNK tv </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> took two curtain calls _ one for himself and one for kerry wood . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cubs # reds # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new chinese foreign minister yang <unk> has sometimes looked ill at ease this week , negotiating his way through the busy corridors of asia 's largest security gathering . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new chinese foreign minister adjusting to limelight at key international meeting </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> striker david di <unk> will be allowed to play friendly matches despite being banned for three months for illegal betting . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> striker david di <unk> allowed to play <unk> despite ban </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: after weeks of uncertainty , democrats in the house of representatives have decided against a confrontation over automobile fuel economy when they take up energy legislation later this week . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: house decides against pursuing tougher fuel economy as part of energy bill </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: web sites usually used by islamic militants groups announced a new , major <unk> video would be released soon . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: major new <unk> video expected on the internet </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> football association wants to issue retrospective bookings for diving this season -- a move <unk> has warned is against the rules of the game . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> fa in dispute with <unk> over issuing retrospective yellow cards to diving players </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> andrew flintoff is expected to be selected for two england cricket squads on monday after recovering from injury . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: andrew flintoff likely to get recall to england squads after recovering from injury </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: olympic gold medalist <unk> <unk> has likely found a new coach -- her older brother . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> s new swimming coach expected to be her older brother </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: gold opened at <unk> <unk> an ounce on wednesday in hong kong , up <unk> <unk> an ounce from <unk> 's close of <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: gold opens higher in hong kong </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> driver and his passenger severely burned in last month 's massive UNK explosion have filed lawsuits against consolidated <unk> , accusing the utility of misconduct . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tow truck driver burned in <unk> steam explosion sues utility company </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the u.s. has cut the population of the <unk> bay detention center to nearly half its peak in <unk> but is struggling to empty it further . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: security human rights concerns make clearing <unk> prison a struggle </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the euro slipped against the dollar on thursday even as u.s. weekly jobless claims rose slightly . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: euro slips against us dollar </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> on thursday criticized the `` excessive use of force '' by american border authorities and slammed what it called <unk> among u.s. legislators who failed to approve immigration reforms . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mexican slams border shooting u.s. failing to pass immigration bill </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a judge ruled friday that <unk> 's state-run health service must review its decision not to give all <unk> 's patients access to three drugs to treat the <unk> disease . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: court says government must review decision to restrict access to <unk> s drugs </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a brief UNK to this weekend 's english premier league games : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: english premier league soccer UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: divers took another body from the wreckage of a collapsed freeway bridge , bringing the death toll to # , with five still missing . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bridge death toll rises to # confirmed as federal authorities promise more money </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results from the second round of the french <unk> soccer league <unk> home teams listed first <unk> : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: french soccer results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: three egyptian brothers died in the explosion of a <unk> land mine they were <unk> with near egypt 's <unk> canal , hospital officials said sunday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: three teenage brothers killed in land mine explosion in egypt </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> euro fell monday against the u.s. dollar after the u.s. commerce department reported consumer spending was up , despite fears the recent housing crisis could trigger a recession . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: euro falls slightly against u.s. dollar </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: six people were fatally shot in the western town of <unk> , a police spokesman said wednesday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: # people fatally shot in germany police report </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: israeli troops uncovered a tunnel constructed by gaza militants who intended to dig under the border fence and into israel , a military spokesman said wednesday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: israeli troops uncover tunnel in northern gaza greenhouse </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the u.s. dollar was trading at <unk> yen at <unk> a.m. <unk> <unk> <unk> wednesday <unk> , down from <unk> yen late wednesday in new york . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: dollar lower against the yen in early <unk> trade </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's children are getting increasingly overweight and unhealthy and the government is asking schools to ban junk food , officials said thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: indian government asks schools to ban junk food </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: max <unk> , a master <unk> whose rhythmic innovations and <unk> provided the <unk> beats that defined <unk> jazz , has died after a long illness . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: max <unk> the master jazz <unk> dead at <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the son of democratic presidential candidate joe <unk> is preparing for deployment to iraq . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: son of anti-war presidential candidate prepares for deployment to iraq next year </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the dollar slipped against the euro on friday after the u.s. federal reserve cut its discount rate to banks by a half percentage point . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: dollar slides against euro as fed cuts discount rate </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: results saturday in the ninth and final stage of tour of germany , a <unk> <unk> <unk> <unk> ride from UNK to <unk> : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tour of germany results </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: at least <unk> people were killed as typhoon <unk> hit the chinese mainland after more than <unk> were evacuated as a precaution , state media reported . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: storms lash chinese coast killing <unk> ; <unk> evacuated as precaution </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: financial giant hsbc said monday it is in talks to buy the controlling stake in korea exchange bank currently held by u.s. private equity group lone star funds . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hsbc in talks to buy controlling stake in korea exchange bank from lone star funds </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> , the <unk> who went to prison as a tax cheat and was <unk> as the `` queen of mean , '' died monday at age <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> <unk> <unk> as queen of mean dies at <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: # . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: u.s. open seeds list </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hurricane dean <unk> into the caribbean coast of <unk> on <unk> as a roaring category # hurricane , lashing ancient <unk> ruins and heading for the modern oil installations . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eye of roaring hurricane dean strikes <unk> s <unk> peninsula </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a detained <unk> academic accused of acting against national security has been released from a <unk> prison after a hefty bail was posted , a top judiciary official said <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> academic held in <unk> released on bail </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> group <unk> league on <unk> reversed itself and called a world war <unk> massacre of <unk> a genocide . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> group in us reverses stand ; calls <unk> massacre genocide </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the body of a <unk> woman was found <unk> in western <unk> , the last of <unk> people who were swept away and killed in a fast-moving river . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: death toll from <unk> flash flood reaches <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the u.s. dollar was mixed against other major currencies in european trading wednesday morning . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us dollar mixed gold down in european morning trading </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: rose UNK , a <unk> who performed <unk> seasons at the metropolitan opera and established herself as a premier voice in american opera , has died . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: rose UNK metropolitan opera star in the <unk> and <unk> dies at <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> plant officials accused of <unk> against dozens of <unk> muslim workers have offered to <unk> break times to help accommodate the workers ' prayer demands . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: swift offers partial accommodation for muslim prayers </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a majority of poles continue to oppose hosting a u.s. missile defense base , according to a poll released friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: poll : <unk> percent of poles oppose hosting u.s. missile defense base </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the german soccer federation voted friday to triple the number of players tested for doping this season from the <unk> tested last season . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: german soccer federation to boost the number of doping controls </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: leading season <unk> in the <unk> after saturday 's third-round games : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: leading season <unk> in the <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> stock average of <unk> issues closed at <unk> points on the <unk> stock exchange monday , up <unk> points , or <unk> percent , from friday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> stock market s main index rises <unk> percent </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the euro drifted lower against the u.s. dollar <unk> after a report showed that german business confidence slipped again , hit by turbulence in global financial markets . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: euro drifts lower on slip in german business sentiment </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: oil and gas futures fell <unk> as concerns about refineries faded and opec suggested that the oil cartel sees no need to boost production . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: oil and gas futures fall on refinery opec news while storm prediction boosts natural gas </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> winds pushed a wildfire closer to sun valley resort 's ski area , while hundreds more homes were ordered evacuated in the valley below . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> winds whip <unk> wildfire near sun valley ski area ; hundreds more homes evacuated </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the u.s. dollar was mostly higher against other major currencies in european trading wednesday morning . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: u.s. dollar mostly higher gold up in european morning trading </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ireland is growing warmer and <unk> because of greenhouse gas emissions and faces increasingly tough challenges to combat floods and manage water supplies , a <unk> report concluded wednesday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ireland growing warmer <unk> because of global warming report finds </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: police used tear gas , water cannons and clubs against demonstrators staging nationwide protests wednesday over government social and economic policies . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: clashes arrests mark national day of protest against economic policies in chile </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> luis <unk> , who managed argentina to the <unk> world cup title , said wednesday he 's full of energy and looking forward to directing <unk> <unk> of <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> looking forward to getting <unk> headed in right direction </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> gay proved thursday he is in a class of his own when he completed a sprint double at the world championships with a win in the <unk> meters . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: gay completes sprint double has eyes on third gold medal </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: an <unk> father who was too drunk to drive had his <unk> son take the wheel , authorities said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: drunk <unk> dad has <unk> year old drive home </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ding UNK believes in <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> rewards strengthen <unk> faith for <unk> believers UNK photos UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> is n't your normal heavyweight boxing champion . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: the people 's champ moves on after controversial victory over <unk> by paul UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in an effort to overcome a growing rift within <unk> , israel 's orthodox chief rabbis met wednesday with reform and conservative leaders . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> leaders meet in israel in effort to overcome religious </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's monthly inflation rate dropped to <unk> percent in <unk> from <unk> percent the previous month , the national bank said friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> inflation drops to <unk> percent in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president <unk> arap moi said rallies planned across the country monday to demand reforms before general elections this year threaten development in <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: moi opposes rallies for reforms </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK , <unk> , grew up in a secular <unk> household in a poor neighborhood of new york city . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> : all true </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: spain 's julian <unk> overcame <unk> UNK of germany in three sets sunday to win the <unk> <unk> <unk> open for his first title as a professional . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> beats UNK for first pro victory </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: deepening economic woes and diplomatic <unk> may force north korean leader kim <unk> il to delay plans to assume full power , a south korean government report said monday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> report : kim <unk> il may delay assuming full power in north korea </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the victorious socialists consulted on <unk> with their political allies over the makeup of a new government and ways to end months of chaos and violence . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> party leaders <unk> over new government composition by UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the leaders of the victorious socialist party consulted with their political allies <unk> on the composition of <unk> 's new government . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> party leaders <unk> over new government composition eds : UNK <unk> graf to clarify UNK t investment schemes caused </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> italian soldier was killed wednesday and three others were wounded in a munitions explosion , the multinational force said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: italian soldier killed three others wounded by blast eds : UNK with new casualties in <unk> graf UNK family and </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the defense force commander who led a mutiny over the government 's plan to hire mercenaries said his dismissal thursday from the army was an act of political <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: stocks close higher in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: rescue workers picked through the rubble of a collapsed school and an office building in northeastern <unk> thursday , desperately trying to save dozens of people buried in an earthquake . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> earthquake in <unk> leaves at least <unk> dead </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a fire raged through a <unk> hotel friday in the beach resort city of <unk> , killing at least <unk> people , including eight <unk> or americans , police said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: at least <unk> dead in hotel fire at beach resort eds : UNK with police saying eight dead may be american or </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: almost <unk> years after the <unk> looted a <unk> oil painting from a <unk> collector in france , the work has been returned to the man 's heirs . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: france returns <unk> painting to <unk> heir eds : <unk> <unk> graf neither UNK ... to UNK with owner 's </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a military court heard more medical testimony sunday in the case of a <unk> soldier accused of killing seven israeli schoolgirls . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: court hears more experts in case involving attack on <unk> by <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the dollar slipped against the yen on profit-taking in early monday trading in <unk> , while share prices rose moderately . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: dollar lower stocks rise in early trading </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the leaders of china 's and vietnam 's ruling communist parties pledged monday to improve relations despite persistent territorial disputes , official chinese media reported . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: vietnam china party chiefs pledge closer ties eds : UNK throughout with official media account of <unk> jiang </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a virgin islands police investigator was charged monday with trying to smuggle <unk> kilograms <unk> <unk> pounds <unk> of cocaine to atlanta in his <unk> luggage , police said . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> police investigator charged with attempted drug smuggling </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> authorities have seized <unk> kilograms <unk> <unk> pounds <unk> of opium in the northern <unk> province , the official islamic republic news agency reported <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> agents seize half a ton of drugs </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> women in leading positions complained about lingering male domination in their society in a meeting <unk> with visiting u.s. first lady hillary <unk> clinton . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> women complain to <unk> clinton about male domination by roland <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: north korean soldiers fired at least <unk> mortar rounds at a south korean border post wednesday during a heavy exchange of gunfire , the defense ministry said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: gunfire exchanged on korean border eds : UNK throughout with details UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: european leaders on wednesday praised <unk> secretary-general <unk> annan 's strict reform plan for the <unk> united nations , and echoed his call for tighter budgets and reduced bureaucracy . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mir 's problems at a glance </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: his third career century _ in a qualifying round match at the <unk> stadium in <unk> on friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: eds : UNK throughout with sri lanka winning </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> is used to missing cuts these days . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> misses another cut UNK shoots <unk> ; <unk> coaches <unk> ; </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: supporters of <unk> <unk> expelled his rival , the <unk> <unk> president , from the ruling party in an effort to close ranks in the face of strong external pressure . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> president booted from ruling party eds : leads throughout with <unk> <unk> president expelled by </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> won his fifth senior <unk> tour event of the year sunday , edging lee <unk> by two strokes to capture the <unk> <unk> million <unk> senior classic . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: results of <unk> national soccer championship games </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: two grenades exploded near a national police station monday , slightly injuring one woman , news reports said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: two grenades explode near spanish police station </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: justin <unk> <unk> high his british open trophy to the delight of cheering fans and friends who turned out at the airport monday to welcome home the <unk> from <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: scores turn out to greet british open winner </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> rico ended water rationing for nearly half a million residents <unk> after heavy rain partly <unk> a reservoir serving the san juan metropolitan area . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> rico ends water rationing </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: red cross negotiators from rivals north korea and south korea held talks wednesday on emergency food shipments to starving north <unk> and agreed to meet again thursday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> meet in <unk> to discuss food aid from south eds : UNK with comment from <unk> official in UNK <unk> ; south </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: grenade blast raises fears of terrorism in <unk> UNK UNK photo UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: arab entrepreneurs should wage a financial `` holy war '' for east <unk> by investing in the arab part of the city , a senior <unk> official says . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> official urges <unk> to invest in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: faced with shifting strategic interests and the need to cut costs , france is <unk> its commitments in africa , a foreign ministry spokesman said friday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: france <unk> its africa policy by <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the chief of staff has endorsed a military court verdict condemning a <unk> soldier to life in prison for killing seven israeli tourist girls , officials and newspapers said sunday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: army chief endorses life sentence on <unk> soldier </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: egypt trounced <unk> <unk> sunday in a group three qualifying round , earning a coveted spot in the <unk> african nations cup . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: egypt <unk> <unk> in african nations cup qualifier </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: several economists say <unk> _ an asian economic success story until <unk> _ could see its first recession in <unk> years . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> : economic woes </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a natural gas tank exploded monday in the kitchen of an oil company building in the urals region , killing at least <unk> people and injuring <unk> , officials said . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: explosion in oil company building kills <unk> eds : UNK to increase death toll to <unk> dead <unk> injured </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: asian stock markets closed generally higher <unk> , but share prices fell in <unk> after rising for three straight sessions . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: asian stock markets close generally higher </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: italian tax police on <unk> searched offices of silvio <unk> 's media company <unk> in connection with a tax evasion investigation of the spanish television network <unk> . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: italian tax police search offices of <unk> group </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: british foreign secretary robin cook met <unk> <unk> opposition leaders wednesday before going on to croatia and talks with president <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cook arrives in <unk> after talks with <unk> by UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> dissident pleaded innocent wednesday to plotting to kill americans in his homeland as his attorney said an earlier agreement to plead guilty was invalid . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> suspect pleads innocent to plotting against americans eds : combines <unk> ; no pickup </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: nasa on wednesday rejected the u.s. astronaut who was supposed to be the next american to live on the russian space station mir because she 's too short . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: european cups scores by the associated press </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the u.s. dollar was traded at <unk> <unk> at <unk> a.m. thursday on the <unk> foreign exchange , down <unk> <unk> from wednesday 's close . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: today in history </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> stock exchange 's main index opened little changed thursday , moving down <unk> points at <unk> on a turnover of <unk> <unk> billion <unk> <unk> <unk> million <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> shares open little changed </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a granddaughter of republic of china founder <unk> sun <unk> said thursday said that the thoughts of her grandfather should be used as the basis for peaceful cross-strait reunification . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK gov t urged to return to three principles of the people </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> UNK <unk> and the <unk> university in southern <unk> signed a memorandum thursday to jointly develop the <unk> and <unk> technologies for UNK use . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> school and u.s. firm to develop UNK technology </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> began to make <unk> checks of passengers on flights arriving from the united states wednesday in a bid to keep the swine flu virus at bay . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> launches <unk> flu checks on flights from north america </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a national human rights museum under the planned ministry of culture is scheduled to open on <unk> # , <unk> , council for cultural affairs chairman <unk> sheng said friday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: human rights museum to open in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> petrochemical giant <unk> plastics group categorically denied monday that the seven members in its core leadership have signed a secret deal to retire by <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> plastics denies secret deal on core leadership </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the u.s. dollar rose against the new <unk> dollar wednesday , adding <unk> <unk> to close at the day 's high of <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: u.s. dollar closes higher on <unk> forex </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> enterprises form oil transport company </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> of a UNK economic cooperation framework agreement <unk> <unk> <unk> between <unk> and china has stirred up a sense of crisis of japan , south korea and <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: united daily news : low tax rates wo n't bring golden decade </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the u.s. dollar rose against the new <unk> dollar thursday , gaining <unk> <unk> to close at <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: u.s. dollar closes higher on <unk> forex </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president chen <unk> lauded <unk> the government 's decision to open <unk> to mainland chinese tourists as a breakthrough in peaceful exchanges with the mainland . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> president lauds <unk> 's opening to mainland chinese tourists </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: last sunday 's boston globe carried an alarming <unk> <unk> article about president bush and the constitution . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK cafeteria </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in heavily <unk> english , the radio <unk> invited <unk> and <unk> to head on over to the `` UNK <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: radio station 's <unk> <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: soccer in italy is nothing short of a religion . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: sport scandal <unk> italian identity </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it 's odd to hear <unk> <unk> , regarded as one of the founding fathers of the internet , to <unk> over UNK books . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: google 's goal : a worldwide web of books </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new on disc : the UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> climbed over the border fence and prepared for the dash into san diego . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: migrants flow into u.s. </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: complaints : the <unk> 's success is also its failure . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: nuts <unk> <unk> : <unk> <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: when gas prices rise , drivers <unk> and politicians talk energy crisis . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> leverage high price of gas </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a washington post article <unk> said that steve <unk> , president of the white house correspondents ' association , works for <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK correction </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the water in the shower went from cold to <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hostel <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in <unk> , seven years into a life sentence for a double homicide , <unk> <unk> sat before a parole board believing that he was never going to be released . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: safeguards for the innocent </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the government filed another round of criminal charges in a widening stock options scandal . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: options scandal widens </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: sometimes it seems the only way to keep children safe from household accidents is constant vigilance . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: how to UNK your home </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: like all <unk> of magic , transforming <unk> of <unk> <unk> <unk> into great red wine is n't easy . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new zealand 's practical magic </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: last week , the international <unk> union stripped <unk> of its official status as a planet . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: # things that like <unk> need a downgrade </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> , the co-founder of a los <unk> social action foundation , had been trying to figure out how to make a statement against terrorism . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bombed bus exhibit to show fearful cost of terrorism </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: shopping for consumer electronics is a pain . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hard questions for hardware </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: our colleague anne hull got lots of reader response on a story last week about an <unk> <unk> discharged from the army for being gay . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: fired <unk> may be back in business </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: today , concern about homework is again reaching a fever pitch . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: it 's not the amount of homework that is the problem </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in the early <unk> , the only thing most people knew about <unk> was that michael <unk> had one . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: let your <unk> do the UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: setting their sights on an operation they say <unk> the internet with a <unk> e-mail messages daily , new york attorney general <unk> <unk> and <unk> corp. . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new york <unk> sue to halt deluge of spam </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> <unk> editors this refers to UNK <unk> lane , post <unk> moved thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK correction </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in <unk> , the u.s. postal service issued two commemorative UNK stamps featuring <unk> the puppy and <unk> the <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: pets on the web </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: as much as they hoped never to hear it , the dreaded news of mad cow disease comes at perhaps the best possible time of year for american <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: timing of mad cow news <unk> impact on <unk> UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK <unk> watches her boys asleep in a <unk> bed . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: keeping together in tough times </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president george <unk> bush 's <unk> <unk> , spot , was old and ailing . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: former head usher at white house witnessed first families at their most vulnerable </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it does n't take a <unk> mind to fret over our state of UNK . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: pop UNK : a possible satire by lee UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: almost <unk> million u.s. homes could wake up saturday to a blank tv screen . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: switch to digital could blank some tv screens saturday </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: two projections offer two contrasting , compelling documents of contemporary urban life . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mark lewis UNK camera </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: she is aware she raised more than a few eyebrows with her music , and that fans might be even more <unk> by her film `` the human contract . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: smith 's new film bears little UNK to tv show </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new <unk> you can watch this weekend : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's alliance with fiat , finalized this month , creates a fresh start for the company and for tens of thousands of workers . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tomorrow 's auto industry </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: fbi agents who raided the office of <unk> william <unk> , <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: fbi threatened to pick lock filing says </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a former chief of staff to <unk> majority whip tom delay , <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> to delay traveled often on others tab </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: is it time for the media to stop <unk> attention on ann <unk> ? UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: promoting <unk> : mainstream media UNK ann <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: if you had a headache , even a <unk> , would you shoot yourself in the head to get rid of it ? UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> comeback plan is a dead end </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: he was a <unk> with missing teeth and flat feet . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: holocaust victim files to be opened </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: former president clinton predicted last week that republican environmental policies are going to lead to more hurricanes . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> must weather hurricane issue </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: anna <unk> must be pleased . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK shark </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> david a. <unk> proposed legislation wednesday to eliminate pay and perks for special district commissioners and put town boards in charge of sanitation districts . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> backs government <unk> plan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: david <unk> first showed signs of chronic fan syndrome in the early <unk> in his native new york city . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: david <unk> the man with obama 's game plan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: at UNK elephant camp , i met boon rot , all <unk> feet and three or so tons of her , as she UNK on bamboo <unk> . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: declining prospects for <unk> 's elephants </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: on its second day of deliberations , a jury <unk> found a <unk> man guilty of stalking `` pulp fiction '' star uma <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: man guilty of stalking <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in brief remarks after meeting with congressional republicans , bush issued his veto threat , complaining that legislation under consideration would `` reward speculators and lenders . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bush vows to veto foreclosure package </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: alexander <unk> <unk> <unk> <unk> , the son of an artist and a painter , not surprisingly was a multimedia artist . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: who is alexander <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the collision occurred shortly before noon <unk> , said <unk> <unk> UNK of the west hollywood sheriff 's station . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: drew <unk> unhurt in <unk> crash </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the rankings for UNK books sold in southern california , as reported by selected book stores : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: best sellers west </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: when kids at UNK singh 's school in <unk> county , <unk> , have a birthday , there are no parents sweeping in with towers of <unk> <unk> with <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a birthday celebration without the <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the deal between government officials and islamic militants in the scenic swat valley could <unk> broader accords with militants in the tribal areas bordering afghanistan . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> signs truce with militant faction </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: but the survey also suggested that the state is moving closer to accepting <unk> marriages , which could create openings for supporters of same-sex marriage as the campaign <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: poll says voters lean toward overturning judges on gay marriage </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: may is a pivotal month for moving and storage companies . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: moving companies hit bumps in economic road </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> <unk> of <unk> <unk> known as el UNK does n't stand out amid the lush UNK <unk> the remote UNK frontier . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hill of ore may hold mountain of conservation woes </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK <unk> bureau UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: $ <unk> UNK looms in <unk> budget </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> long after a recent practice , magic center <unk> <unk> <unk> in the stands and <unk> with team general manager <unk> smith about life . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: as <unk> <unk> mount so does frustration </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ext goes here . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> head goes UNK test </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> as usual , rays manager joe <unk> had his reasons for starting willy <unk> ahead of pat <unk> at designated hitter on thursday against <unk> <unk> ryan UNK . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> starts because ... </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK <unk> bureau UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a lavish but limited look </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: this is the time of year when people often take golf lessons . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a lesson about lessons </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> i 've heard that cow <unk> can be used for energy production , but not human waste . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: on not wasting waste </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> winds through lush <unk> along a los <unk> river walk surrounded by native plants and birds . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: land issue no walk in the park </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: when she founded the digital ad agency UNK nine years ago , kat egan was n't worried about finding good talent . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a digital boot camp to <unk> talent for agencies </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: one by one the favorites have fallen : the <unk> , the capitals , the red wings , the devils . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: regular season was irrelevant in the east </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the boxer amir khan spent late april and early may in <unk> , british columbia , his entry into the united states delayed while he obtained a work visa . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: after verbal sparring a lopsided title fight </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> movie critic san <unk> <unk> , copyright <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: commentary mick <unk> on movies age in the eye of the UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: never mind the plot . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: make <unk> s style <unk> even without a <unk> big </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: maybe it takes a shock <unk> like <unk> the love <unk> <unk> to get alex sink to relax . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: sink s lips loosen on radio show </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: using certain <unk> drugs for years at a time can help keep some types of cancer in <unk> longer , doctors reported thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: study finds drugs can keep some cancer in <unk> longer </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> , outgoing and accustomed to working , <unk> <unk> wants to earn a <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: lack of aid for child care pushes some to welfare </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: stocks declined monday , dragged down by <unk> losses in the financial sector , which has been mired in uncertainty over financial reforms and european debt troubles . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: financial reform and europe s debt send shares lower </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: consider it the ultimate girl 's night out . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: why we UNK fashion s <unk> four </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> a. <unk> ; <unk> pages ; $ <unk> <unk> reviewed by peter <unk> san <unk> <unk> , copyright <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: his finest hour <unk> s war : <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: internal documents from bp show that there were serious problems and safety concerns with the <unk> horizon rig far earlier than those the company described to congress last week . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK UNK trims <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK <unk> los <unk> <unk> - column on who should win the western conference semifinals between the <unk> and san antonio spurs . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: los <unk> daily news budget </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: new executive director of basketball operations <unk> <unk> said he would not speak with the players until after the <unk> finished their playoff run . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> pushed to brink as nets cruise in game # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> for release sunday , may <unk> <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK for release sunday may <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: long before the final horn blew wednesday at the <unk> center , the mavericks felt they never had a legitimate chance at beating the san antonio spurs . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: technical knockout : nelson harris booted early in <unk> game # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: john e. UNK , the former chief executive and president of the <unk> corp. , died on may <unk> at <unk> UNK in <unk> , <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: john e. UNK former chief of <unk> corp. dies </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the master plan for ground zero is <unk> , which is not necessarily bad news . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: for ground zero disarray <unk> and an opportunity awaits </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: if ever there was a classic case of `` no free lunch , '' popular pain control medications are it . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: personal health : <unk> of pain relief often hide in tiny type </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: during his first two days on the witness stand , l. <unk> <unk> appeared calm as he <unk> a series of tough questions . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: things turn a bit <unk> at trial of <unk> chief </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the independent political groups known as <unk> should be required to disclose their funding sources while election campaigns are in progress , three republican house members said wednesday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bill would require quick disclosure of <unk> groups funding </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: get ready to think about office politics and family affairs in UNK terms again . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: living with wolves : wolf society alpha to <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: if you walk into the office of a scientist , chances are you 'll see a white board hanging on the wall , covered with UNK . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: profile : michael UNK explores where brain meets mind </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the curse at first - reversed . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: red sox over a 's <unk> red UNK 's # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: numbers help define <unk> <unk> , in atlanta this week to defend the <unk> <unk> charity championship . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> : <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a wealthy <unk> pays a publishing house to issue the novel he has written : a classic example of a <unk> project . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> of UNK : the conductor as composer as entrepreneur </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: mike <unk> has n't quite rescued the nba from itself yet , but the phoenix coach said that the suns ' success this season <unk> his approach to the game . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: suns showing no signs of slowing down in playoffs </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: for <unk> <unk> , a single word changed everything . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a ceremonial event <unk> into a wedding </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: `` i bring you a taste of my UNK , '' says our host as my friend <unk> and i sit down to dinner beside a <unk> fire . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: is le UNK the next <unk> maybe </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> should n't be here . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hats off to <unk> leukemia victim </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it was supposed to be a cozy affair . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: out in force : as # beloved sci-fi franchises fade away fans have seized the spotlight like never before </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the new york times said in an editorial for friday , may <unk> : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: editorial : the <unk> of afghanistan </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the raiders were definitely up for dan marino 's first appearance ever at the <unk> <unk> sunday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tough defense stood up to marino 's challenge </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: not long ago , in a drug and alcohol rehabilitation center in texas , a <unk> boy was <unk> withdrawal at its worst . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: stuck on the web : the symptoms of internet addiction </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> stocks rose for the first time in seven days , led by retailer UNK , amid optimism over the economy for <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> stocks rise on economic recovery UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the new york times said in an editorial on wednesday , <unk> # : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: editorial : <unk> picks a judge </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: texas senator takes seat on influential <unk> panel UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> eds : removes state in <unk> to correct to u.s. senate </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: here 's the routine : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: breaking small rules to preserve the big ones </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> said on thursday that clinton had also been impressed by <unk> 's <unk> negotiating style at the united nations . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: washington : into diplomacy </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the new york times said in an editorial on friday , <unk> # : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: editorial : cabinet renewal </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: his <unk> empire now stretches from sea to shining sea , which means that <unk> smith 's next acquisition might be a navy . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: some see <unk> smith 's <unk> empire as a growing threat to </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it got so bad for the <unk> dolphins on sunday that <unk> were even being directed at their coach , jimmy <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: giants over <unk> it 's true : guaranteed </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: general growth properties said it bought three michigan shopping malls from an investment group called UNK properties for $ <unk> million in cash , debt and securities . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: general growth buys # malls for $ <unk> mln from investment group </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> inc. 's mutual fund subsidiary said it hired marketing executive per UNK from <unk> investments . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's fund group hires per UNK from <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: if you 've ever been to <unk> , <unk> , you would not likely think of it as a high-tech haven . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: tech report : UNK <unk> screen <unk> <unk> to top of the </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> investments said it hired marketing executive kirk <unk> from <unk> rival <unk> investments to serve as national sales manager for its bank wholesale group . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> hires marketing executive kirk <unk> from <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in a broad preview of his <unk> plans , president clinton promised wednesday to govern with the same centrist themes he used to win re-election . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: clinton promises a <unk> coalition of center </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: international business machines corp. won a jury verdict that its <unk> did n't cause a <unk> operator to suffer hand and wrist injuries . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> wins UNK lawsuit week after dec loss UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: recognizing that information technology is the fastest-growing segment of the UNK industry , computer task group is also preparing to provide <unk> computer consultants . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK : savvy <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: if you want to buy a faithful companion for the animal lover on your holiday shopping list , here are a few suggestions . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK gift guide : page by page </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> co. , the world 's biggest UNK manufacturer , has a <unk> percent share of china 's <unk> drinks market , the company said , citing two market surveys . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> claims <unk> % share of china 's UNK market </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it was one of the greatest collections of <unk> talent ever to gather on ice _ and not skate . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: solo but not alone </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the yankees lost one and did not gain one monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: yankees continue pursuit of wells </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: starts of new u.s. housing rebounded sharply in <unk> , government figures showed , as a dip in borrowing costs encouraged hesitant buyers to come back into the market . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: u.s. economy : housing starts jump <unk> % in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: christmas week might not seem the best time to open a movie that celebrates a millionaire <unk> as an american folk hero . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a free speech hero it 's not that simple </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: whoever says toys are n't educational has n't been shopping lately . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: think of messages toys send </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: young said he 's inclined to go along with a sentencing agreement proposed by the government and <unk> 's attorneys . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> judge indicates prison term of <unk> months UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: general electric co. said it plans to split its stock <unk> , increase its dividend and buy back more shares because it 's churning out record amounts of cash . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ge proposes <unk> stock split raises dividend <unk> % UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: `` my fellow americans , '' a comedy , is rated <unk> for <unk> language and a sex scene involving <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: my fellow americans stumbles into lame <unk> old men <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: `` my fellow americans '' could have been called `` <unk> old <unk> '' _ and probably should have been . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: fellow americans : save your vote </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: michael has a lot of problems . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: message to michael : gay life as a sitcom </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> it happens every year ; my <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> smoke : it UNK every winter </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president fidel <unk> , who was hospitalized for the second time in <unk> days over the weekend , may need heart surgery , his spokesman said . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> president hospitalized may need heart surgery </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: is croatia 's autocratic president at death 's door ? or is he , as his aides say , bursting with renewed UNK bounce ? UNK </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK for croatia 's <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's $ <unk> billion takeover bid for <unk> <unk> is the most dramatic step yet in the consolidation of the u.s. military industry . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: make way for the united states of <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it 's here . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> too much of a good thing </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: `` fear not : for <unk> , i bring you good <unk> of great joy . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a century of christian <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: so your <unk> inadvertently sliced off your little toe , and you think you 're going to sue . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: arizona <unk> bills revisit <unk> territory ; suits would be <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> UNK is in the <unk> , waiting for her date with destiny . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: when UNK <unk> its tail </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the south african reserve bank will likely reduce its key lending rate at least twice in <unk> as growth slows , analysts said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: south african central bank may cut rates twice : outlook <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the new york times said in an editorial on friday , <unk> <unk> : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: editorial : <unk> open the phone market </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it was hot in the summer of <unk> , too hot for <unk> to maintain their normal conception rates , according to the nation 's <unk> farmers . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> seeks accuracy with <unk> and pig count : commodity views </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: when he <unk> friday morning and looked at the sky , arizona state football coach bruce <unk> had a decision to make : should he be <unk> or smart ? UNK </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: rain sends sun devils to ballroom </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: turmoil at the top often means bad news for an entire professional sports organization , but there 's one team that was glad to see a shakeup upstairs . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK keen on blues coaching move </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> is a <unk> sort of UNK superstar , not as good looking as tom <unk> or as polished as peter <unk> or as edgy as dan rather . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a reporter 's life : <unk> <unk> _ UNK but <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> at the turn of the century had a master plan for <unk> , parks and <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> : has been lost </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> to buy <unk> 's insurance arm for $ <unk> bln UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK marshall 's christmas wish was <unk> simple . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a wonderful gift </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK , ohio . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: commentary : public displays of what </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: for a week , you have eyed that <unk> box of cheap cookies that a business associate sent over . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> start over thanks to food pros </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in life 's rich pageant , few <unk> are as <unk> as watching congress spend the public 's treasure . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: grand old spending party </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: left-hander mike <unk> will not be on the rangers ' postseason roster unless he displays a dramatic improvement in his ability to field his position . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> could be left off rangers postseason roster </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK : <unk> 's <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the sale of sprint corp. casts a huge shadow over one of the largest commercial construction projects in the united states . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> plans to buy sprint cast doubt sprint construction in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: these columns for release thursday , <unk> # , <unk> are moving today to clients of the new york times news service . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cox news service commentary budget </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: los <unk> might have lost its bid for an expansion team to houston on wednesday , but fans who like to watch football and the tv networks were winners . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: no <unk> team good for tv viewers </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the good news came the other day in the form of a phone call . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK : printing the good news </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: `` rising interest rates are bad news for banks . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: bank stocks and the jinx of fed credit tightening </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: find many gaps UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: supporters say bush 's record rosy ; critics </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: one hundred pitches . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> made one mistake that hurt </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the blueprint for beating the new england patriots has been drawn and future opponents surely will try to follow it . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chiefs revealed patriots weaknesses </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: state farm mutual automobile insurance co. may need to buy more insurance for itself , to cover such disasters as the one that heavily damaged this <unk> company last week . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: getting what they paid for </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the divorce is final . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: delta ends alliance with <unk> to concentrate on air france </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: with deadlines for college applications looming and a UNK on <unk> , <unk> <unk> of <unk> was getting nervous . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: easing stress when children UNK to colleges </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in addition , forecasts , even when they are right , are no longer enough for most clients . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new york : specific markets </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: already the <unk> <unk> have <unk> the high country with snow , already taken the edge off the colors that swell in the <unk> below the <unk> . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: the season of the hunt </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: how a company lets its cash talk </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: update : scored seven goals and three assists in kings ' first five games , all on the road . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> with kings UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: when it comes to personal finance software , mac users do n't have much choice . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> upgrade a bit <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> , it 's just the same old stuff . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: every little thing sting does is no longer magic </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: brian shaw 's <unk> has been <unk> to say the least . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> add guard depth sign shaw </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: confession time . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: looking back pope paul vi was right on contraception </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it blew up fast . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: in postseason nobody better than <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: does the white house or congress have the right to send american troops into battle ? UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: house members challenge clinton 's authority to bomb <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in `` american beauty , '' kevin <unk> 's dissatisfied suburban husband <unk> <unk> UNK his ambitious wife , <unk> <unk> <unk> <unk> <unk> , for her <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: movies put <unk> in focus </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: these sports stories for release wednesday , <unk> <unk> , <unk> , are moving today to clients of the new york times news service . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cox news service sports budget </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: having trouble finding a particular item ? we 'll try to help you locate it . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: where to get hard to find items </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: four men sit at a table playing cards . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: wedding black professionals relationships </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: eds : <unk> for sunday , <unk> <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> program has design on community </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: back in the <unk> , when the united states had prohibition , quite a few <unk> grew rich running booze over the border to UNK their neighbors . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK madness : british columbia goes to pot </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: you know the pose , <unk> fans . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: odom making impression </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in each city the angels visit , manager terry <unk> hears the same questions from reporters . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: angels feast on east </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: i 've been expecting grocery stores to start selling computers in the <unk> line any day now . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK <unk> intriguing but you probably wo n't purchase one </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> made an unusual choice three years ago when it tapped john <unk> <unk> as superintendent of public schools . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ex-general fights for schools and for his life </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: after five months of upheaval and disarray , the dodgers already might have their first disaster for the <unk> season . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: after loss <unk> makes demand </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the weather was cool during <unk> tech 's football practice monday , but coach george <unk> was hot and <unk> after the workout . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> gets defensive after loss to bc </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK <unk> and three friends traveled from argentina to new york for a quick getaway . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: commentary : over a bridge the real life of <unk> york </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it all started <unk> enough , with a dozen UNK containers of dried <unk> and <unk> in a plastic <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: how to make the most of <unk> UNK without letting them </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> UNK <unk> out of bed not long after some people are turning off <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mother teacher UNK as an elite <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: every once in a while a book comes along that <unk> a national debate , and i 'm not talking about special prosecutor <unk> <unk> 's report to congress . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: proving the merits of affirmative action </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: for these reasons , the impeachment process must be <unk> and deliberate . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: washington : elected him </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> are not directly involved in most of the action in soccer , especially on successful teams . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: revolution keep playoff hopes afloat </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the new u.s. open singles champion said she walked into her hotel room late saturday afternoon and found the message light bright as <unk> 's nose . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's u.s. open title fills void in women 's tennis </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: these sports stories for release <unk> , <unk> <unk> , <unk> , are moving today to clients of the new york times news service . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cox news service <unk> sports budget </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: a preview of what could be the best fall in years </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> airline and delta air lines announced two differing strategies to upgrade their long-haul <unk> service for business travelers . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: business travel : competing strategies ; crowded skies </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK free art ! UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: the mature <unk> : </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a former fbi agent said wednesday he will not <unk> turn over to federal officials documents he retrieved from james <unk> ray 's car <unk> years ago . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: justice <unk> agent at odds over king clue </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: fall <unk> and <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: home video ; UNK videos this fall ; other new releases </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: this year a community group donated money for her school fees , a school uniform and shoes . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> : <unk> clothes </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the day that <unk> <unk> dumped his report on congress , the governor of texas sat in his office , striving <unk> to keep the conversation on his re-election race . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: george bush counters the UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it is no longer a question of <unk> ; skip away demonstrated again saturday that he is great . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: skip away runs away in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: manager terry <unk> got angry last week when he thought his angels overlooked the tampa bay devil rays before heading into their series with the texas rangers . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: angels set the stage with victory </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: about <unk> years ago , people living in what is now southern <unk> camped on the pacific shore and <unk> on fish , <unk> and <unk> . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: in <unk> evidence of an early human maritime culture </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president clinton may or may not escape impeachment , but the path to high office has been drastically changed by his ordeal . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: analysis : impeachment fever may <unk> presidential campaigns </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> government has questioned former president <unk> about his personal wealth and accusations that he had billions of dollars in overseas bank accounts , officials said on <unk> . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> wealth under inquiry </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK free art ! UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: shopping around </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: i sat in my <unk> hut , watching a <unk> circle the lone light <unk> and trying to <unk> myself to take a shower . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: missing the <unk> of home </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a dark and stormy night . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: killer <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: nearly <unk> regional artists will <unk> on the <unk> <unk> next weekend for the third annual celebration of fine craft . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> times in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: one play , one pitch to <unk> evans was all bobby <unk> was wishing for against the washington <unk> saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> beats washington <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> came into the season fearful that his impending free agency might be a distraction . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: batting title for <unk> with free agency ahead </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: seemingly harmless surveys , contests and electronic pen pal web sites are sometimes <unk> disguised tools directed at youngsters to obtain marketing data , children 's rights advocates charge . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: congress considers regulating children 's marketing on world </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: students who spend too much time trying to learn their lessons at computers may end up with lower scores on tests , according to a study released <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: study says computers may hinder student performance </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: what do you think of <unk> <unk> ? is the <unk> center fielder <unk> ? should he be included on the list of great players who have worn <unk> ? UNK </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: prime time for <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> , <unk> _ a part of the massachusetts landscape since colonial times , the business of growing <unk> has in some ways changed very little since then . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: firm finds niche in UNK data for <unk> growers </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the legend of <unk> UNK is just beginning . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> in semifinal but it s <unk> that falls </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: sometimes , the best coaching tactic is a desperate plea to a team 's best player . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> UNK second straight title </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: mixing judgment and UNK against <unk> grows as the public gains a higher awareness of the injuries , both physical and psychological , suffered by young victims . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: mixing judgment and restraint UNK for use the boston globe sexual misconduct at # years old </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: when it comes to health care , mitt <unk> is hoping to have it both ways , even as he accuses the white house of doing the same . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> on health care : a particular spin </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: it is the year of the big man in the nfl draft . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: advance for sunday UNK <unk> the top <unk> players in <unk> nfl draft </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: st. <unk> UNK was one of the interesting group of <unk> that <unk> ross gathered around him at the new <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: st. <unk> UNK and the new <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: they rose <unk> from a <unk> <unk> , two <unk> UNK that <unk> over the spectacular tidal <unk> of <unk> island . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: torn UNK : <unk> island <unk> duo now just a lone tree </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: cape district attorney <unk> by grand jury UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: grand jury <unk> UNK da </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: on <unk> , <unk> <unk> , the senate minority leader , called for the abolition of municipal fire departments . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: commentary : the fire next time </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK <unk> bureau UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> in admiration as he campaigns on veto </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: questions should be directed to alan gordon at <unk> or e-mail UNK . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cox UNK budget for <unk> UNK <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the first-round playoff series last year between the boston <unk> and the chicago bulls was a memorable <unk> of <unk> and clutch shots . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> with <unk> lead : debated but rarely done </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> UNK , a diabetes physician who works north of <unk> , has a chilling story to tell about a man who came into a clinic in rural <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: the chronic disease dilemma <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: confidence in greek assets sank to a new low monday , as chancellor <unk> <unk> of germany kept up the pressure on greece , insisting on tougher austerity measures . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: confidence in greek debt sinks again matthew <unk> reported from paris </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: i like to look in people 's windows . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: life <unk> : confessions of a <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: st. petersburg with <unk> <unk> ready to go back behind the plate , the rays added another pitcher to their bullpen by calling up veteran <unk> <unk> <unk> . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> called up to <unk> bullpen </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: for much of this decade , the <unk> of palm and <unk> were <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: how the UNK of <unk> and palm grew UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: this city is in the <unk> of olympic fever . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: as UNK pile up olympic pride <unk> a bit for <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: fidel <unk> turns <unk> on wednesday as brother <unk> tackles a monumental challenge : keeping the revolution alive despite a widening generation gap . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: life after fidel : UNK rising prices <unk> in cuba but rebellion is scarce </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> 's <unk> anchor leg at the water <unk> helped propel nbc to the largest audience ever recorded for the first sunday of any olympic games broadcast . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: initial <unk> tv ratings are record breaking </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: lake <unk> vista <unk> white might not win the job as the <unk> ' starting left defensive end . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: next goal starts for a start </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: did you realize the chinese communist party was that much into cute ? UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: commentary : i m UNK in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> is n't here to wave a flag , or pledge allegiance . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: sports column : <unk> looking for a little personal <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: almost everything i was told about running in <unk> was wrong . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: an <unk> <unk> in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the night of the gun , by david <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: new and recommended the boston globe </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> -- for years , the standard treatment for patients with blood <unk> in <unk> deep in a <unk> has been blood <unk> that stop the <unk> from getting bigger . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK uses sound waves to speed UNK UNK uses sound waves to speed treatment </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the margin of victory seemed almost impossible , his finishing time a UNK moment . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> bolt a UNK record </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> , poor <unk> , we knew him well . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> in <unk> # catches laughs but few breaks </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: one of china 's top sports officials said sunday that a simple paperwork error was to blame for an age controversy involving the chinese women 's gymnastics team . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chinese official blames age issue on UNK errors </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: for most shoppers , it 's second nature to punch in a personal identification number when using a debit card to pay for a purchase . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: john c. UNK <unk> ; pioneer in credit industry </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the caucus : UNK <unk> 's non <unk> UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: convention notes and news </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> lama is exhausted . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> lama citing exhaustion cancels trips </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> got the job done on saturday , adding a grade i win in <unk> to his long and impressive list of accomplishments . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> edges long shot to win the <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> - the eyes are hidden . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: murphy 's law : seek diversity </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: colin l. <unk> said nothing -- a silence that spoke volumes to many in the white house on thursday morning . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: in meeting with former officials bush defends iraq policy </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: another new year , same old promises : i will ease off my favorite barbecue joint 's <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: saving $ <unk> in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a supreme court victory won by california <unk> is now starting to resemble a fine <unk> : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: high court ruling aside wine still not flowing across state lines </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: `` monk , '' a simple idea that has proven surprisingly resilient -- a <unk> genius <unk> quirky crimes -- resumes its fourth season tonight with a particularly silly case . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: monk keeps things <unk> quirky </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: cox news service <unk> , ohio -- <unk> coles has played in and coached more than <unk> basketball games . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: forty years later texas western 's watershed victory still vivid </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: st. <unk> - it 's been so long now - eight years - that the gold medal seems as if it happened to someone else . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> treasures golden memories </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> - an icon to some and a sinister <unk> to others , a priest with strong <unk> ties currently sits languishing in a <unk> jail with suspected leukemia . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> jail # months no charges </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> UNK , a former <unk> linebacker with the giants , was happy shaping the defensive line that would become the <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: youngest coach in nfl history still gives advice </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: after spending <unk> years in prison for a killing he did n't commit , ken marsh finally is poised to get some <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: $ <unk> for <unk> years wrongly held in prison </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> pat <unk> , it seems , is n't the only one who thinks he can see god 's purpose in natural disasters . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: editorial : blaming god for disasters </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> lopez , who works at the state <unk> , is sitting in a <unk> red <unk> chair on the floor of the senate chamber . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: profile : UNK institution </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: never mind that she has dark blond hair and light blue eyes and the <unk> of skin . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: she writes from the heart </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: james franco , best known as harry <unk> , peter parker 's best friend in the `` <unk> '' movies , is not your usual romantic lead . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK UNK founders in a sea of <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: democrats in <unk> and <unk> , borrowing an idea usually advanced by conservative republicans , are promoting bible classes in the public schools . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: democrats in # southern states push bills on bible study </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: cox news service <unk> , texas -- my boyfriend <unk> to me the other day that he has become an <unk> widow . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK for sunday <unk> <unk> <unk> an <unk> <unk> turned <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: something you 're not likely to hear at the start of a ski or <unk> group lesson : you 're going to fall down . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: how do you ski or <unk> practice practice falling and practice </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> - immediately following saturday afternoon 's game at the <unk> center , the lightning boarded a speeding train to washington for today 's game against the capitals . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: on track and rolling with rout </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> government said today substantial progress has been made in its peace talks with the leftist national democratic front <unk> <unk> <unk> held last month in the netherlands . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: weather forecast for major world cities </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> beat <unk> <unk> in a world cup asian zone group one qualifying match in <unk> , <unk> arabia , on monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> beats <unk> <unk> in world cup qualifier </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chinese <unk> <unk> <unk> met here today with his <unk> counterpart UNK UNK , and his party . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chinese <unk> meets <unk> visitors </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> is considering making a modest contribution to the <unk> multinational protection force set up to secure relief aid operations in <unk> , the italian defense ministry said today . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> may contribute to protection force to <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> patrick <unk> apologized on wednesday for being drunk while playing for his national team in the a davis cup . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> UNK for being drunk in davis cup game </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the security council today <unk> to the general assembly the nominations of <unk> candidates for judges of the international criminal tribunal for the former <unk> . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> results at <unk> open tennis <unk> in hong kong </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: government troops has <unk> an attack by rebel forces along <unk> 's eastern borders , the spokesman of the <unk> armed forces said today . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: government forces repel rebel attack in eastern </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the european union <unk> eu <unk> and the united states today reached an agreement on a dispute over the <unk> act against cuba , u.s. officials confirmed . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us confirms deal on <unk> law dispute </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chinese <unk> and foreign minister <unk> <unk> met with <unk> deputy foreign minister UNK <unk> here today . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: chinese <unk> foreign ministers meet in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> will <unk> its currency <unk> to offset problems likely to affect its national economy , southern african broadcasting association <unk> saba <unk> reported today . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> to <unk> national currency </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: five workers of the <unk> airline will be tried to determine the degree of their responsibility in an airliner accident in <unk> <unk> which killed <unk> persons . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: market exchange rates </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the opposition labor party 's lead has dropped to its lowest level for four years , according to the latest UNK poll for the sunday times . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: uk labor 's lead drops to lowest for four years </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president <unk> <unk> has stated that <unk> has a `` very special relationship with china and the bilateral ties have been tested by time . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> has very special relationship with china : </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> <unk> olympic games will probably have traffic jams and transport problems , according to ioc official . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> faces transport problem </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> <unk> has invested # billion baht <unk> about <unk> million us dollars <unk> to build a new plant in east <unk> , which officially opened today . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: key <unk> markets fixed rates </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> , china 's largest industrial city will add <unk> supermarkets and <unk> chain stores this year to its <unk> service outlets of this kind . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: results of table tennis world championships UNK # </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the democratic people 's republic of korea <unk> south korea in the women 's team semi-finals at the world table tennis championships here on sunday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> korea sails into women 's team final </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: two passenger trains crashed today in <unk> china 's <unk> province , killing <unk> people and leaving many others injured . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: train crash kills <unk> in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: president george <unk> bush met with visiting <unk> prime minister <unk> <unk> in the white house on wednesday to discuss issues on bilateral relations and cooperation in fighting terror . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: us president meets <unk> pm in white house </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> prime minister <unk> <unk> on friday reiterated his confidence about the <unk> economy , saying prospects are good that japan will resolve the issue of bad loans around <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> reiterates confidence in UNK economy </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: by feng jian , yang UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK analysis : eu conference launches <unk> of <unk> marathon </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> share prices closed only <unk> percent higher monday in listless trading as the market continued to consolidate . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> stocks end slightly higher </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: nigeria will support south africa 's <unk> world cup bid and go all out for the <unk> edition itself , <unk> president <unk> <unk> said here monday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: nigeria targets to host <unk> world cup : <unk> president </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the <unk> governing council <unk> rejected an expected <unk> offer of sending <unk> troops to iraq , <unk> al <unk> tv channel reported . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK : iraq 's governing council rejects <unk> troops deployment : tv </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: costa rica has ruled out the possibility of opening up its telecommunication sector in spite of pressure <unk> by the united states , reports from guatemala city said wednesday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: costa rica rules out opening telecommunication sector </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: government and business leaders from <unk> central and east european countries are gathering in <unk> , <unk> , for the region 's first investment summit on <unk> <unk> to <unk> . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: first investment summit to be held for central east europe </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: ministerial officials attending the forum for economic and trade cooperation between china and <unk> countries , which opened here on sunday , dedicated the morning debate to three topics . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: forum between china <unk> countries focuses on three <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: vietnam will start the construction of its first underground railway in the ho chi <unk> city in mid - <unk> with an investment of <unk> million us dollars . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: vietnam to build first underground railways </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: top interpol officers on wednesday asked its members to devise rules and procedures for policing at the global level and providing legal status to red corner notices against wanted fugitives . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: interpol asks world <unk> to make rules for global policing </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chinese permanent representative to the united nations wang <unk> on wednesday urged the un and the international community to continue supporting <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: china stresses continued international support for <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: despite generous donation from the international community , millions of people will still wake up hungry on the world food day which falls on thursday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> urges more UNK to hungry people </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president <unk> <unk> has reiterated support for <unk> president <unk> <unk> de <unk> , who is facing a UNK crisis at home , a <unk> diplomat said thursday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: ecuador 's gov t reiterates support for <unk> <unk> president </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the visiting <unk> arabian crown prince <unk> bin <unk> aziz sunday called on <unk> to move on a path of unity and tolerance . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> first division <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: israel 's air forces on monday launched a third missile strike on gaza city in less than four hours , <unk> witnesses said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: israel launches third airstrike on gaza city </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: following are the world cup alpine world cup champions in the <unk> season : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: men 's world cup ski champions last season </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> police captured <unk> kg of heroin in two separate operations , the <unk> telegraph agency reported wednesday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> police seize <unk> kg heroin </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> minister of foreign affairs <unk> <unk> will visit china on <unk> <unk> , announced foreign ministry spokeswoman <unk> <unk> here thursday at a regular press conference . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> foreign minister to visit china </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: sri <unk> prime minister <unk> <unk> will meet us president george bush to review the country 's current <unk> peace progress , officials said . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: sri <unk> pm to meet us president to review peace process </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's top shares prices ended higher monday as rumors indicating a swift sell-off concerning state-run oil major <unk> state oil <unk> <unk> <unk> . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> stocks close up </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: south korean stocks continued their winning streak on foreign and program buying , with the main index bouncing back to the <unk> level on <unk> . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: s. korean stocks continue rising </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> president <unk> <unk> wednesday spoke highly of the <unk> relations , saying that the relations between the two countries are always excellent , strategic and permanent . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: interview : <unk> relations are excellent strategic permanent : <unk> by rong UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: following are the schedule for the world cup volleyball from <unk> # in <unk> <unk> cities : UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: schedule of world cup volleyball UNK women </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: truck drivers and mechanics working for waste management inc. returned to work thursday after a two-week strike in los <unk> , organizers said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: trash workers end strike in los <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: hosts <unk> won the men 's soccer title by beating <unk> <unk> <unk> here at the <unk> chinese city games on friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: hosts <unk> wins men 's soccer title at chinese city games </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: in anticipation of strong winds returning , fire departments across southern california are bracing for a possible new round of fresh wildfires , authorities said on friday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> : firefighters gear up for fresh fires in s. california </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: chief of army staff of <unk> general <unk> <unk> on saturday declared a state of emergency in the country , the state-run <unk> reported . </s> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> ld : <unk> declares state of emergency in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: strengthened border patrol has led to a <unk> drop in arrests of undocumented migrants this year at the <unk> frontier , it was reported on wednesday . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: arrests of <unk> drop </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> 's biggest mining company said it will spill out # billion u.s. dollars for its three-year expansion scheme , daily mail reported on friday . </s> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> 's biggest miner to spend # bln dollars on expansion </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a landslide in northwest china 's <unk> province has trapped three people since saturday , the local government said on sunday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: landslide UNK three in northwest china </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: six people were confirmed dead and two others were hospitalized in a food poisoning case on sunday in central china 's <unk> province , local government sources said on monday . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: major news items in leading indian UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> <unk> was reported for training with <unk> on <unk> with a twisted ankle , team doctor jose luis <unk> said . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> trains with ankle injury for <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> fishing vessel , which has been captured off the <unk> coast in may and released early this month , arrived at <unk> port of <unk> on wednesday . </s> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> fishing vessel released off <unk> coast arrives at <unk> <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> stocks opened sharply lower friday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: major news items in leading new zealand UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: coach of manchester city football club <unk> <unk> signed three thai young players to the english premiership league team at a ceremony in <unk> friday morning . </s> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: UNK man </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: china 's sun hui claimed gold in the women 's <unk> <unk> at the <unk> world <unk> championships here on saturday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: china 's sun wins women 's <unk> of <unk> at world <unk> championships </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> has imported about <unk> tons of maize from <unk> and more tons are lined up for <unk> , according sunday mail . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: head-to-head record for <unk> masters cup final </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: the death toll from the last thursday night 's cyclone in <unk> shot up to <unk> till <unk> p. m. monday , according to the food and disaster management ministry . </s>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: cyclone death toll in <unk> rises to <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: nigeria 's federal capital <unk> on monday got the hosting right of the sixth students for the <unk> of global <unk> <unk> <unk> <unk> world cup tournament . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: nigeria to host <unk> <unk> world cup </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: expensive scientific instruments are now available for public rental throughout china via a new <unk> database launched on thursday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: foreign exchange rates in <unk> </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: <unk> and <unk> will cooperate in transporting <unk> oil to european consumer markets , the <unk> and <unk> presidents stressed in a joint press statement on thursday . </s> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> to cooperate in transporting oil </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: between <unk> and <unk> people in vietnam die of diseases caused by smoking each year , according to local newspaper youth on monday . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> <unk> <unk> die from smoking annually </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: by <unk> writer jiang UNK , <unk> UNK UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> UNK focus : china france sign <unk> nuclear energy deal </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: a <unk> <unk> earthquake rocked metro manila at around <unk> p.m. <unk> but there is no immediate reports of casualties . </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> ld UNK : earthquake hits metro manila </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: UNK </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: <unk> ld : <unk> 's <unk> steps down as army chief </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | source: jing <unk> , commander of the second artillery force of the chinese people 's liberation army <unk> <unk> <unk> , wrapped up his official visit to sweden wednesday . </s> <pad> <pad>\n",
      " | prediction: <s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s> </s>\n",
      " | target: senior chinese military official UNK up visit to sweden </s> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad> <pad>\n",
      " | ROUGE Score:\n",
      " | RG-1(F):     0.09\n",
      " | RG-2(F):     0.00\n",
      " | RG-L(F):     0.09\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import pickle\n",
    "import time\n",
    "\n",
    "import mindspore.nn as nn\n",
    "import mindspore.common.dtype as mstype\n",
    "from mindspore.ops import operations as P\n",
    "from mindspore.common.tensor import Tensor\n",
    "from mindspore.train.model import Model\n",
    "from mindspore.train.serialization import load_checkpoint, load_param_into_net\n",
    "\n",
    "from src.utils import Dictionary\n",
    "from src.transformer.transformer_for_infer import TransformerInferModel\n",
    "from src.transformer.transformer_for_train import TransformerTraining\n",
    "from src.utils.load_weights import load_infer_weights\n",
    "from src.utils.rouge_score import rouge\n",
    "\n",
    "class TransformerInferCell(nn.Cell):\n",
    "    def __init__(self, network):\n",
    "        super(TransformerInferCell, self).__init__(auto_prefix=False)\n",
    "        self.network = network\n",
    "\n",
    "    def construct(self,\n",
    "                  source_ids,\n",
    "                  source_mask):\n",
    "        predicted_ids, predicted_probs = self.network(source_ids,\n",
    "                                                      source_mask)\n",
    "        return predicted_ids, predicted_probs\n",
    "\n",
    "def get_rouge_score(result, vocab):\n",
    "    \"\"\"Calculate ROUGE score.\"\"\"\n",
    "    predictions = []\n",
    "    targets = []\n",
    "    for sample in result:\n",
    "        predictions.append(' '.join([vocab[t] for t in sample['prediction']]))\n",
    "        targets.append(' '.join([vocab[t] for t in sample['target']]))\n",
    "        print(f\" | source: {' '.join([vocab[t] for t in sample['source']])}\")\n",
    "        print(f\" | prediction: {predictions[-1]}\")\n",
    "        print(f\" | target: {targets[-1]}\")\n",
    "\n",
    "    return rouge(predictions, targets)\n",
    "\n",
    "config = config()\n",
    "config.epoch = 1 #for test\n",
    "#这个文件名称要修改为训练好的模型文件路径\n",
    "#config.checkpoint_file_path = \"./output/checkpoint/ckpt_ft/ft-1_1266.ckpt\"\n",
    "\n",
    "vocab_path = './vocab/all_en.dict.bin'\n",
    "vocab = Dictionary.load_from_persisted_dict(vocab_path)\n",
    "config.vocab_size = vocab.size\n",
    "\n",
    "eval_dataset = load_dataset(input_file=config.test_dataset,\n",
    "                            batch_size=config.batch_size,\n",
    "                            epoch_count=1,\n",
    "                            sink_mode=config.dataset_sink_mode,\n",
    "                            shuffle=False)\n",
    "\n",
    "tfm_model = TransformerInferModel(config=config, use_one_hot_embeddings=False)\n",
    "tfm_model.init_parameters_data()\n",
    "\n",
    "params = tfm_model.trainable_params()\n",
    "weights = load_infer_weights(config)\n",
    "\n",
    "for param in params:\n",
    "    value = param.data\n",
    "    name = param.name\n",
    "    with open(\"weight_after_deal.txt\", \"a+\") as f:\n",
    "        weights_name = name\n",
    "        f.write(weights_name + \"\\n\")\n",
    "        if isinstance(value, Tensor):\n",
    "            if weights_name in weights:\n",
    "                assert weights_name in weights\n",
    "                param.set_data(Tensor(weights[weights_name], mstype.float32))\n",
    "\n",
    "print(\" | Load weights successfully.\")\n",
    "\n",
    "tfm_infer = TransformerInferCell(tfm_model)\n",
    "model = Model(tfm_infer)\n",
    "\n",
    "predictions = []\n",
    "probs = []\n",
    "source_sentences = []\n",
    "target_sentences = []\n",
    "for batch in eval_dataset.create_dict_iterator(output_numpy=True, num_epochs=1):\n",
    "    source_sentences.append(batch[\"source_eos_ids\"])\n",
    "    target_sentences.append(batch[\"target_eos_ids\"])\n",
    "\n",
    "    source_ids = Tensor(batch[\"source_eos_ids\"], mstype.int32)\n",
    "    source_mask = Tensor(batch[\"source_eos_mask\"], mstype.int32)\n",
    "\n",
    "    start_time = time.time()\n",
    "    predicted_ids, entire_probs = model.predict(source_ids, source_mask)\n",
    "    print(f\" | Batch size: {config.batch_size}, \"\n",
    "          f\"Time cost: {time.time() - start_time}.\")\n",
    "\n",
    "    predictions.append(predicted_ids.asnumpy())\n",
    "    probs.append(entire_probs.asnumpy())\n",
    "\n",
    "output = []\n",
    "for inputs, ref, batch_out, batch_probs in zip(source_sentences,\n",
    "                                               target_sentences,\n",
    "                                               predictions,\n",
    "                                               probs):\n",
    "    for i in range(config.batch_size):\n",
    "        if batch_out.ndim == 3:\n",
    "            batch_out = batch_out[:, 0]\n",
    "\n",
    "        example = {\n",
    "            \"source\": inputs[i].tolist(),\n",
    "            \"target\": ref[i].tolist(),\n",
    "            \"prediction\": batch_out[i].tolist(),\n",
    "            \"prediction_prob\": batch_probs[i].tolist()\n",
    "        }\n",
    "        output.append(example)\n",
    "\n",
    "with open(config.output, \"wb\") as f:\n",
    "    pickle.dump(output, f, 1)\n",
    "\n",
    "score = get_rouge_score(output, vocab)\n",
    "print(score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "source": [
    "## 3.7 评估指标\n",
    "\n",
    "### 3.7.1 PPL\n",
    "对于自然语言生成模型，最重要的问题就是生成的文本序列是否符合我们人类的使用习惯。\n",
    "在自然语言处理领域中，最常用的评估指标为PPL，perplexity（困惑度），它可以衡量语言模型的好坏，计算方法是根据每个词来估计一句话出现的概率，并用句子长度作标准化。\n",
    "\n",
    "$$\n",
    "PPL(S) = P(w_1w_2...w_N)^{-\\frac{1}{N}} \\\\\n",
    " = \\sqrt[N]{\\frac{1}{p(w_1 w_2...w_N)}} \\\\\n",
    " = \\sqrt[N]{\\Pi^N_{i=1} \\frac{1}{p(w_i | w_1w_2...w_{i-1})}}\n",
    "$$\n",
    "\n",
    "其中$S$代表输入的sentence，$N$为句子长度，$p(w_i)$是第$i$个词的概率。第一个词就是 $p(w_1|w_0)$，而$w_0$表示句子的起始，是个占位符。\n",
    "\n",
    "PPL越小，$p(w_i)$则越大，我们期望的sentence出现的概率就越高。Perplexity可以认为是average branch factor（平均分支系数），即预测下一个词时可以有多少个合理的选择，可选词数越少，表示生成的句子越接近目标，模型越准确。但perplexity只是大致估计下训练效果，它不是完全意义上的标准，具体问题还是要具体分析。\n",
    "\n",
    "### 3.7.2 ROUGE\n",
    "ROUGE指标是在机器翻译、自动摘要、问答生成等领域常见的评估指标，全称是 (Recall-Oriented Understudy for Gisting Evaluation)。ROUGE通过将模型生成的摘要或者回答与参考答案（一般是人工生成的）进行比较计算，得到对应的得分。ROUGE指标与BLEU指标非常类似，均可用来衡量生成结果和标准结果的匹配程度，不同的是ROUGE基于召回率，BLEU更看重准确率。\n",
    "\n",
    "在文本摘要微调任务上，使用 Gigaword 测试集计算ROUGE-1、ROUGE-2 和 ROUGE-L 的 F1 分数作为微调模型的评估结果。\n",
    "\n",
    "ROUGE-1和ROUGE-2属于ROUGE-N算法，就是将模型生成的结果和标准结果按N-gram拆分后，计算召回率。公式为：\n",
    "$$\n",
    "ROUGE-N = \\frac{\\sum_{S \\in ReferenceSummaries} \\sum_{gram_n \\in S} Count_{match}gram_n}{\\sum_{S \\in ReferenceSummaries} \\sum_{gram_n \\in S}Count(gram_n)}\n",
    "$$\n",
    "\n",
    "分子表示所有样本按N-gram拆分后与生成的结果按N-gram拆分后匹配上个数的和；分母表示所有样本的标准结果，按N-gram拆分后的和。\n",
    "\n",
    "Rouge-L的L意为: Longest Common Subsequence，最长公共子序列。Rouge-L的公式可以表示为：\n",
    "$$\n",
    "R_{lcs} = \\frac{LCS(X, Y)}{m}\\\\\n",
    "P_{lcs} = \\frac{LCS(X, Y)}{n}\\\\\n",
    "F_{lcs} = \\frac{(1+\\beta^2)R_{lcs}P_{lcs}}{R_{lcs}+\\beta^2 P_{lcs}}\n",
    "$$\n",
    "\n",
    "公式中的$F_{lcs}$就是ROUGE-L的得分。\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore",
   "language": "python",
   "name": "mindspore"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}