Skip to content

Commit 0df401c

Browse files
authored
v0.8.6
- add IFM and DIFM model - add FEFM and DeepFEFM model - add PositionEncoding Layer - add training param to activation layer - improve the compatibility for tensorflow 2.5 - fix error when linear_feature_columns is empty in tensorflow 1.12
2 parents a94a8ec + d5f6647 commit 0df401c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+833
-116
lines changed

.github/ISSUE_TEMPLATE/bug_report.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ assignees: ''
88
---
99

1010
**Describe the bug(问题描述)**
11-
A clear and concise description of what the bug is.
11+
A clear and concise description of what the bug is.Better with standalone code to reproduce the issue.
1212

1313
**To Reproduce(复现步骤)**
1414
Steps to reproduce the behavior:
@@ -18,9 +18,9 @@ Steps to reproduce the behavior:
1818
4. See error
1919

2020
**Operating environment(运行环境):**
21-
- python version [e.g. 3.5, 3.7]
22-
- tensorflow version [e.g. 1.4.0, 1.15.0, 2.4.0]
23-
- deepctr version [e.g. 0.8.3,]
21+
- python version [e.g. 3.6, 3.7]
22+
- tensorflow version [e.g. 1.4.0, 1.15.0, 2.5.0]
23+
- deepctr version [e.g. 0.8.6,]
2424

2525
**Additional context**
2626
Add any other context about the problem here.

.github/ISSUE_TEMPLATE/question.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,5 @@ Add any other context about the problem here.
1616

1717
**Operating environment(运行环境):**
1818
- python version [e.g. 3.6]
19-
- tensorflow version [e.g. 1.4.0, 1.5.0, 2.4.0]
20-
- deepctr version [e.g. 0.8.3,]
19+
- tensorflow version [e.g. 1.4.0, 1.15.0, 2.5.0]
20+
- deepctr version [e.g. 0.8.6,]

.github/workflows/ci.yml

+5-3
Original file line numberDiff line numberDiff line change
@@ -14,15 +14,17 @@ jobs:
1414
build:
1515

1616
runs-on: ubuntu-latest
17-
timeout-minutes: 120
17+
timeout-minutes: 180
1818
strategy:
1919
matrix:
2020
python-version: [3.6,3.7]
21-
tf-version: [1.4.0,1.15.0,2.1.0,2.4.0]
21+
tf-version: [1.4.0,1.15.0,2.1.0,2.5.0]
2222

2323
exclude:
2424
- python-version: 3.7
2525
tf-version: 1.4.0
26+
- python-version: 3.7
27+
tf-version: 1.15.0
2628

2729
steps:
2830

@@ -40,7 +42,7 @@ jobs:
4042
pip install -q requests
4143
pip install -e .
4244
- name: Test with pytest
43-
timeout-minutes: 120
45+
timeout-minutes: 180
4446
run: |
4547
pip install -q pytest
4648
pip install -q pytest-cov

README.md

+6-3
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111

1212
[![Documentation Status](https://readthedocs.org/projects/deepctr-doc/badge/?version=latest)](https://deepctr-doc.readthedocs.io/)
1313
![CI status](https://github.com/shenweichen/deepctr/workflows/CI/badge.svg)
14-
[![Coverage Status](https://coveralls.io/repos/github/shenweichen/DeepCTR/badge.svg?branch=master)](https://coveralls.io/github/shenweichen/DeepCTR?branch=master)
14+
[![codecov](https://codecov.io/gh/shenweichen/DeepCTR/branch/master/graph/badge.svg)](https://codecov.io/gh/shenweichen/DeepCTR)
1515
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/d4099734dc0e4bab91d332ead8c0bdd0)](https://www.codacy.com/gh/shenweichen/DeepCTR?utm_source=github.com&utm_medium=referral&utm_content=shenweichen/DeepCTR&utm_campaign=Badge_Grade)
1616
[![Disscussion](https://img.shields.io/badge/chat-wechat-brightgreen?style=flat)](./README.md#DisscussionGroup)
1717
[![License](https://img.shields.io/github/license/shenweichen/deepctr.svg)](https://github.com/shenweichen/deepctr/blob/master/LICENSE)
@@ -55,7 +55,10 @@ Let's [**Get Started!**](https://deepctr-doc.readthedocs.io/en/latest/Quick-Star
5555
| FiBiNET | [RecSys 2019][FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction](https://arxiv.org/pdf/1905.09433.pdf) |
5656
| FLEN | [arxiv 2019][FLEN: Leveraging Field for Scalable CTR Prediction](https://arxiv.org/pdf/1911.04690.pdf) |
5757
| BST | [DLP-KDD 2019][Behavior sequence transformer for e-commerce recommendation in Alibaba](https://arxiv.org/pdf/1905.06874.pdf) |
58+
| IFM | [IJCAI 2019][An Input-aware Factorization Machine for Sparse Prediction](https://www.ijcai.org/Proceedings/2019/0203.pdf) |
5859
| DCN V2 | [arxiv 2020][DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://arxiv.org/abs/2008.13535) |
60+
| DIFM | [IJCAI 2020][A Dual Input-aware Factorization Machine for CTR Prediction](https://www.ijcai.org/Proceedings/2020/0434.pdf) |
61+
| FEFM and DeepFEFM | [arxiv 2020][Field-Embedded Factorization Machines for Click-through rate prediction](https://arxiv.org/abs/2009.09931) |
5962

6063
## Citation
6164

@@ -78,7 +81,7 @@ If you find this code useful in your research, please cite it using the followin
7881
## DisscussionGroup
7982

8083
- [Discussions](https://github.com/shenweichen/DeepCTR/discussions)
81-
- 公众号:**浅梦的学习笔记**
84+
- 公众号:**浅梦学习笔记**
8285
- wechat ID: **deepctrbot**
8386

8487
![wechat](./docs/pics/code.png)
@@ -112,7 +115,7 @@ If you find this code useful in your research, please cite it using the followin
112115
</td>
113116
<td>
114117
​ <a href="https://github.com/TanTingyi"><img width="70" height="70" src="https://github.com/TanTingyi.png?s=40" alt="pic"></a><br>
115-
<a href="https://github.com/TanTingyi">LeoCai</a>
118+
<a href="https://github.com/TanTingyi">Tan Tingyi</a>
116119
<p> Chongqing University <br> of Posts and <br> Telecommunications </p>​
117120
</td>
118121
</tr>

deepctr/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
from .utils import check_version
22

3-
__version__ = '0.8.5'
3+
__version__ = '0.8.6'
44
check_version(__version__)

deepctr/estimator/models/__init__.py

+1
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,4 @@
1010
from .pnn import PNNEstimator
1111
from .wdl import WDLEstimator
1212
from .xdeepfm import xDeepFMEstimator
13+
from .deepfefm import DeepFEFMEstimator

deepctr/estimator/models/deepfefm.py

+89
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# -*- coding:utf-8 -*-
2+
"""
3+
Author:
4+
Harshit Pande
5+
6+
Reference:
7+
[1] Field-Embedded Factorization Machines for Click-through Rate Prediction]
8+
(https://arxiv.org/abs/2009.09931)
9+
10+
"""
11+
12+
import tensorflow as tf
13+
14+
from ..feature_column import get_linear_logit, input_from_feature_columns
15+
from ..utils import DNN_SCOPE_NAME, deepctr_model_fn, variable_scope
16+
from ...layers.core import DNN
17+
from ...layers.interaction import FEFMLayer
18+
from ...layers.utils import concat_func, add_func, combined_dnn_input, reduce_sum
19+
20+
21+
def DeepFEFMEstimator(linear_feature_columns, dnn_feature_columns,
22+
dnn_hidden_units=(128, 128), l2_reg_linear=0.00001, l2_reg_embedding_feat=0.00001,
23+
l2_reg_embedding_field=0.00001, l2_reg_dnn=0, seed=1024, dnn_dropout=0.0,
24+
dnn_activation='relu', dnn_use_bn=False, task='binary', model_dir=None,
25+
config=None, linear_optimizer='Ftrl', dnn_optimizer='Adagrad', training_chief_hooks=None):
26+
"""Instantiates the DeepFEFM Network architecture or the shallow FEFM architecture (Ablation support not provided
27+
as estimator is meant for production, Ablation support provided in DeepFEFM implementation in models
28+
29+
:param linear_feature_columns: An iterable containing all the features used by linear part of the model.
30+
:param dnn_feature_columns: An iterable containing all the features used by deep part of the model.
31+
:param dnn_hidden_units: list,list of positive integer or empty list, the layer number and units in each layer of DNN
32+
:param l2_reg_linear: float. L2 regularizer strength applied to linear part
33+
:param l2_reg_embedding_feat: float. L2 regularizer strength applied to embedding vector of features
34+
:param l2_reg_embedding_field: float, L2 regularizer to field embeddings
35+
:param l2_reg_dnn: float. L2 regularizer strength applied to DNN
36+
:param seed: integer ,to use as random seed.
37+
:param dnn_dropout: float in [0,1), the probability we will drop out a given DNN coordinate.
38+
:param dnn_activation: Activation function to use in DNN
39+
:param dnn_use_bn: bool. Whether use BatchNormalization before activation or not in DNN
40+
:param task: str, ``"binary"`` for binary logloss or ``"regression"`` for regression loss
41+
:param model_dir: Directory to save model parameters, graph and etc. This can
42+
also be used to load checkpoints from the directory into a estimator
43+
to continue training a previously saved model.
44+
:param config: tf.RunConfig object to configure the runtime settings.
45+
:param linear_optimizer: An instance of `tf.Optimizer` used to apply gradients to
46+
the linear part of the model. Defaults to FTRL optimizer.
47+
:param dnn_optimizer: An instance of `tf.Optimizer` used to apply gradients to
48+
the deep part of the model. Defaults to Adagrad optimizer.
49+
:param training_chief_hooks: Iterable of `tf.train.SessionRunHook` objects to
50+
run on the chief worker during training.
51+
:return: A Tensorflow Estimator instance.
52+
"""
53+
54+
def _model_fn(features, labels, mode, config):
55+
train_flag = (mode == tf.estimator.ModeKeys.TRAIN)
56+
57+
linear_logits = get_linear_logit(features, linear_feature_columns, l2_reg_linear=l2_reg_linear)
58+
final_logit_components = [linear_logits]
59+
60+
with variable_scope(DNN_SCOPE_NAME):
61+
sparse_embedding_list, dense_value_list = input_from_feature_columns(features, dnn_feature_columns,
62+
l2_reg_embedding=l2_reg_embedding_feat)
63+
64+
fefm_interaction_embedding = FEFMLayer(
65+
regularizer=l2_reg_embedding_field)(concat_func(sparse_embedding_list, axis=1))
66+
67+
fefm_logit = tf.keras.layers.Lambda(lambda x: reduce_sum(x, axis=1, keep_dims=True))(
68+
fefm_interaction_embedding)
69+
70+
final_logit_components.append(fefm_logit)
71+
72+
if dnn_hidden_units:
73+
dnn_input = combined_dnn_input(sparse_embedding_list, dense_value_list)
74+
dnn_input = concat_func([dnn_input, fefm_interaction_embedding], axis=1)
75+
76+
dnn_output = DNN(dnn_hidden_units, dnn_activation, l2_reg_dnn, dnn_dropout, dnn_use_bn, seed=seed)(
77+
dnn_input, training=train_flag)
78+
79+
dnn_logit = tf.keras.layers.Dense(
80+
1, use_bias=False, kernel_initializer=tf.keras.initializers.glorot_normal(seed))(dnn_output)
81+
82+
final_logit_components.append(dnn_logit)
83+
84+
logits = add_func(final_logit_components)
85+
86+
return deepctr_model_fn(features, mode, logits, labels, task, linear_optimizer, dnn_optimizer,
87+
training_chief_hooks=training_chief_hooks)
88+
89+
return tf.estimator.Estimator(_model_fn, model_dir=model_dir, config=config)

deepctr/estimator/utils.py

+15-6
Original file line numberDiff line numberDiff line change
@@ -44,27 +44,29 @@ def _eval_metric_ops(self,
4444
_summary_key(self._name, "prediction/mean"): metrics.mean(predictions, weights=weights),
4545
_summary_key(self._name, "label/mean"): metrics.mean(labels, weights=weights),
4646
}
47-
tf.summary.scalar("prediction/mean", metric_ops[_summary_key(self._name, "prediction/mean")][1])
48-
tf.summary.scalar("label/mean", metric_ops[_summary_key(self._name, "label/mean")][1])
47+
48+
summary_scalar("prediction/mean", metric_ops[_summary_key(self._name, "prediction/mean")][1])
49+
summary_scalar("label/mean", metric_ops[_summary_key(self._name, "label/mean")][1])
50+
4951

5052
mean_loss = losses.compute_weighted_loss(
5153
unweighted_loss, weights=1.0, reduction=losses.Reduction.MEAN)
5254

5355
if self._task == "binary":
5456
metric_ops[_summary_key(self._name, "LogLoss")] = metrics.mean(mean_loss, weights=weights, )
55-
tf.summary.scalar("LogLoss", mean_loss)
57+
summary_scalar("LogLoss", mean_loss)
5658

5759
metric_ops[_summary_key(self._name, "AUC")] = metrics.auc(labels, predictions, weights=weights)
58-
tf.summary.scalar("AUC", metric_ops[_summary_key(self._name, "AUC")][1])
60+
summary_scalar("AUC", metric_ops[_summary_key(self._name, "AUC")][1])
5961
else:
6062

6163
metric_ops[_summary_key(self._name, "MSE")] = metrics.mean_squared_error(labels, predictions,
6264
weights=weights)
63-
tf.summary.scalar("MSE", mean_loss)
65+
summary_scalar("MSE", mean_loss)
6466

6567
metric_ops[_summary_key(self._name, "MAE")] = metrics.mean_absolute_error(labels, predictions,
6668
weights=weights)
67-
tf.summary.scalar("MAE", metric_ops[_summary_key(self._name, "MAE")][1])
69+
summary_scalar("MAE", metric_ops[_summary_key(self._name, "MAE")][1])
6870

6971
return metric_ops
7072

@@ -206,3 +208,10 @@ def to_float(x, name="ToFloat"):
206208
return tf.to_float(x, name)
207209
except AttributeError:
208210
return tf.compat.v1.to_float(x, name)
211+
212+
213+
def summary_scalar(name, data):
214+
try:
215+
tf.summary.scalar(name, data)
216+
except AttributeError: # tf version 2.5.0+:AttributeError: module 'tensorflow._api.v2.summary' has no attribute 'scalar'
217+
tf.compat.v1.summary.scalar(name, data)

deepctr/feature_column.py

+12-6
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,15 @@
1+
import tensorflow as tf
12
from collections import namedtuple, OrderedDict
23
from copy import copy
34
from itertools import chain
45

56
from tensorflow.python.keras.initializers import RandomNormal, Zeros
6-
from tensorflow.python.keras.layers import Input
7+
from tensorflow.python.keras.layers import Input, Lambda
78

89
from .inputs import create_embedding_matrix, embedding_lookup, get_dense_input, varlen_embedding_lookup, \
910
get_varlen_pooling_list, mergeDict
1011
from .layers import Linear
11-
from .layers.utils import concat_func, add_func
12+
from .layers.utils import concat_func
1213

1314
DEFAULT_GROUP_NAME = "default_group"
1415

@@ -145,7 +146,7 @@ def build_input_features(feature_columns, prefix=''):
145146

146147

147148
def get_linear_logit(features, feature_columns, units=1, use_bias=False, seed=1024, prefix='linear',
148-
l2_reg=0):
149+
l2_reg=0, sparse_feat_refine_weight=None):
149150
linear_feature_columns = copy(feature_columns)
150151
for i in range(len(linear_feature_columns)):
151152
if isinstance(linear_feature_columns[i], SparseFeat):
@@ -166,16 +167,21 @@ def get_linear_logit(features, feature_columns, units=1, use_bias=False, seed=10
166167
if len(linear_emb_list[i]) > 0 and len(dense_input_list) > 0:
167168
sparse_input = concat_func(linear_emb_list[i])
168169
dense_input = concat_func(dense_input_list)
170+
if sparse_feat_refine_weight is not None:
171+
sparse_input = Lambda(lambda x: x[0] * tf.expand_dims(x[1], axis=1))(
172+
[sparse_input, sparse_feat_refine_weight])
169173
linear_logit = Linear(l2_reg, mode=2, use_bias=use_bias, seed=seed)([sparse_input, dense_input])
170174
elif len(linear_emb_list[i]) > 0:
171175
sparse_input = concat_func(linear_emb_list[i])
176+
if sparse_feat_refine_weight is not None:
177+
sparse_input = Lambda(lambda x: x[0] * tf.expand_dims(x[1], axis=1))(
178+
[sparse_input, sparse_feat_refine_weight])
172179
linear_logit = Linear(l2_reg, mode=0, use_bias=use_bias, seed=seed)(sparse_input)
173180
elif len(dense_input_list) > 0:
174181
dense_input = concat_func(dense_input_list)
175182
linear_logit = Linear(l2_reg, mode=1, use_bias=use_bias, seed=seed)(dense_input)
176-
else:
177-
# raise NotImplementedError
178-
return add_func([])
183+
else: #empty feature_columns
184+
return Lambda(lambda x: tf.constant([[0.0]]))(list(features.values())[0])
179185
linear_logit_list.append(linear_logit)
180186

181187
return concat_func(linear_logit_list)

deepctr/layers/__init__.py

+16-11
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@
55
from .interaction import (CIN, FM, AFMLayer, BiInteractionPooling, CrossNet, CrossNetMix,
66
InnerProductLayer, InteractingLayer,
77
OutterProductLayer, FGCNNLayer, SENETLayer, BilinearInteraction,
8-
FieldWiseBiInteraction, FwFMLayer)
8+
FieldWiseBiInteraction, FwFMLayer, FEFMLayer)
99
from .normalization import LayerNormalization
1010
from .sequence import (AttentionSequencePoolingLayer, BiasEncoding, BiLSTM,
11-
KMaxPooling, SequencePoolingLayer,WeightedSequenceLayer,
12-
Transformer, DynamicGRU)
13-
from .utils import NoMask, Hash,Linear,Add,combined_dnn_input
11+
KMaxPooling, SequencePoolingLayer, WeightedSequenceLayer,
12+
Transformer, DynamicGRU,PositionEncoding)
13+
14+
from .utils import NoMask, Hash, Linear, Add, combined_dnn_input, softmax, reduce_sum
1415

1516
custom_objects = {'tf': tf,
1617
'InnerProductLayer': InnerProductLayer,
@@ -36,12 +37,16 @@
3637
'KMaxPooling': KMaxPooling,
3738
'FGCNNLayer': FGCNNLayer,
3839
'Hash': Hash,
39-
'Linear':Linear,
40+
'Linear': Linear,
4041
'DynamicGRU': DynamicGRU,
41-
'SENETLayer':SENETLayer,
42-
'BilinearInteraction':BilinearInteraction,
43-
'WeightedSequenceLayer':WeightedSequenceLayer,
44-
'Add':Add,
45-
'FieldWiseBiInteraction':FieldWiseBiInteraction,
46-
'FwFMLayer': FwFMLayer
42+
'SENETLayer': SENETLayer,
43+
'BilinearInteraction': BilinearInteraction,
44+
'WeightedSequenceLayer': WeightedSequenceLayer,
45+
'Add': Add,
46+
'FieldWiseBiInteraction': FieldWiseBiInteraction,
47+
'FwFMLayer': FwFMLayer,
48+
'softmax': softmax,
49+
'FEFMLayer': FEFMLayer,
50+
'reduce_sum': reduce_sum,
51+
'PositionEncoding':PositionEncoding
4752
}

deepctr/layers/core.py

+5-2
Original file line numberDiff line numberDiff line change
@@ -189,8 +189,11 @@ def call(self, inputs, training=None, **kwargs):
189189

190190
if self.use_bn:
191191
fc = self.bn_layers[i](fc, training=training)
192-
193-
fc = self.activation_layers[i](fc)
192+
try:
193+
fc = self.activation_layers[i](fc, training=training)
194+
except TypeError as e: # TypeError: call() got an unexpected keyword argument 'training'
195+
print("make sure the activation function use training flag properly", e)
196+
fc = self.activation_layers[i](fc)
194197

195198
fc = self.dropout_layers[i](fc, training=training)
196199
deep_input = fc

0 commit comments

Comments
 (0)