adapt.instance_based.TrAdaBoost

class adapt.instance_based.TrAdaBoost(estimator=None, Xt=None, yt=None, n_estimators=10, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]

Transfer AdaBoost for Classification

TrAdaBoost algorithm is a supervised instances-based domain adaptation method suited for classification tasks.

The method is based on a “reverse boosting” principle where the weights of source instances poorly predicted decrease at each boosting iteration whereas the ones of target instances increase.

The algorithm performs the following steps:

1. Normalize weights: $\sum w_S + \sum w_T = 1$.
2. Fit an estimator $f$ on source and target labeled data $(X_S, y_S), (X_T, y_T)$ with the respective importances weights: $w_S, w_T$.
3. Compute error vectors of training instances:
- $\epsilon_S = L_{01}(f(X_S), y_S)$.
- $\epsilon_T = L_{01}(f(X_T), y_T)$.
4. Compute total weighted error of target instances: $E_T = \frac{1}{n_T} w_T^T \epsilon_T$.
5. Update source and target weights:
- $w_S = w_S \beta^{\epsilon_S}$.
- $w_T = w_T \beta_T^{-\epsilon_T}$.
Where:
- $\beta = 1 \setminus (1 + \sqrt{2 \text{ln} n_S \setminus N})$.
- $\beta_T = E_T \setminus (1 - E_T)$.
6. Return to step 1 and loop until the number $N$ of boosting iteration is reached.

The prediction are then given by the vote of the $N \setminus 2$ last computed estimators weighted by their respective parameter $\beta_T$.

Parameters

estimatorsklearn estimator or tensorflow Model (default=None): Estimator used to learn the task. If estimator is None, a LinearRegression instance is used as estimator.
Xtnumpy array (default=None): Target input data.
ytnumpy array (default=None): Target output data.
n_estimatorsint (default=10): Number of boosting iteration.
lrfloat (default=1.): Learning rate. For higher lr, the sample weights are updating faster.
copyboolean (default=True): Whether to make a copy of estimator or not.
verboseint (default=1): Verbosity level.
random_stateint (default=None): Seed of random generator.
paramskey, value arguments: Arguments given at the different level of the adapt object. It can be, for instance, compile or fit parameters of the estimator or kernel parameters etc… Accepted parameters can be found by calling the method _get_legal_params(params).

See also

TrAdaBoostR2, TwoStageTrAdaBoostR2

References

1: [1] Dai W., Yang Q., Xue G., and Yu Y. “Boosting for transfer learning”. In ICML, 2007.

Examples

>>> from sklearn.linear_model import RidgeClassifier
>>> from adapt.utils import make_classification_da
>>> from adapt.instance_based import TrAdaBoost
>>> Xs, ys, Xt, yt = make_classification_da()
>>> model = TrAdaBoost(RidgeClassifier(), n_estimators=10, Xt=Xt[:10], yt=yt[:10], random_state=0)
>>> model.fit(Xs, ys)
Iteration 0 - Error: 0.2550
Iteration 1 - Error: 0.2820
Iteration 2 - Error: 0.3011
Iteration 3 - Error: 0.3087
Iteration 4 - Error: 0.3046
Iteration 5 - Error: 0.2933
Iteration 6 - Error: 0.2819
Iteration 7 - Error: 0.2747
Iteration 8 - Error: 0.2712
Iteration 9 - Error: 0.2698
>>> model.score(Xt, yt)
0.66

Attributes

estimators_list of object: List of fitted estimators
estimator_errors_1D array of float: Array of weighted estimator errors computed on labeled target data.
estimator_weights_1D array of float: Array of estimator importance weights.
sample_weights_src_list of numpy arrays: List of source sample weight for each iteration.
sample_weights_tgt_list of numpy arrays: List of target sample weight for each iteration.

Methods

`__init__`([estimator, Xt, yt, n_estimators, ...])
`fit`(X, y[, Xt, yt, sample_weight_src, ...])	Fit TrAdaBoost
`fit_estimator`(X, y[, sample_weight, ...])	Fit estimator on X, y.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`predict`(X)	Return weighted vote of estimators.
`predict_estimator`(X, **predict_params)	Return estimator predictions for X.
`predict_weights`([domain])	Return sample weights.
`score`(X, y)	Return the TrAdaboost score on X, y.
`set_fit_request`(*[, sample_weight_src, ...])	Request metadata passed to the `fit` method.
`set_params`(**params)	Set the parameters of this estimator.
`set_predict_request`(*[, domain])	Request metadata passed to the `predict` method.
`set_score_request`(*[, domain, sample_weight])	Request metadata passed to the `score` method.
`unsupervised_score`(Xs, Xt)	Return unsupervised score.

__init__(estimator=None, Xt=None, yt=None, n_estimators=10, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]

fit(X, y, Xt=None, yt=None, sample_weight_src=None, sample_weight_tgt=None, **fit_params)[source]

Fit TrAdaBoost

Parameters

Xnumpy array: Source input data.
ynumpy array: Source output data.
Xtarray (default=None): Target input data. If None, the Xt argument given in init is used.
ytarray (default=None): Target input data. If None, the Xt argument given in init is used.
sample_weight_srcnumpy array, (default=None): Initial sample weight of source data
sample_weight_tgtnumpy array, (default=None): Initial sample weight of target data
fit_paramskey, value arguments: Arguments given to the fit method of the estimator.

Returns

selfreturns an instance of self

fit_estimator(X, y, sample_weight=None, random_state=None, warm_start=True, **fit_params)[source]

Fit estimator on X, y.

Parameters

Xarray: Input data.
yarray: Output data.
sample_weightarray: Importance weighting.
random_stateint (default=None): Seed of the random generator
warm_startbool (default=True): If True, continue to fit estimator_, else, a new estimator is fitted based on a copy of estimator. (Be sure to set copy=True to use warm_start=False)
fit_paramskey, value arguments: Arguments given to the fit method of the estimator and to the compile method for tensorflow estimator.

Returns

estimator_fitted estimator

get_metadata_routing()[source]

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns

routingMetadataRequest: A MetadataRequest encapsulating routing information.

get_params(deep=True)[source]

Get parameters for this estimator.

Parameters

deepbool, default=True: Not used, here for scikit-learn compatibility.

Returns

paramsdict: Parameter names mapped to their values.

predict(X)[source]

Return weighted vote of estimators.

Parameters

Xarray: Input data.

Returns

y_predarray: Vote results.

predict_estimator(X, **predict_params)[source]

Return estimator predictions for X.

Parameters

Xarray: input data

Returns

y_predarray: prediction of estimator.

predict_weights(domain='src')[source]

Return sample weights.

Return the final importance weighting.

You can secify between “source” and “target” weights with the domain parameter.

Parameters

domainstr (default=”tgt”): Choose between "source", "src" and "target", "tgt".

Returns

weightssource sample weights

score(X, y)[source]

Return the TrAdaboost score on X, y.

Parameters

Xarray: input data
yarray: output data

Returns

scorefloat: estimator score.

set_fit_request(*, sample_weight_src: Union[bool, None, str] = '$UNCHANGED$', sample_weight_tgt: Union[bool, None, str] = '$UNCHANGED$') → adapt.instance_based._tradaboost.TrAdaBoost[source]

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weight_srcstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for sample_weight_src parameter in fit.
sample_weight_tgtstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for sample_weight_tgt parameter in fit.

Returns

selfobject: The updated object.

set_params(**params)[source]

Set the parameters of this estimator.

Parameters

**paramsdict: Estimator parameters.

Returns

selfestimator instance: Estimator instance.

set_predict_request(*, domain: Union[bool, None, str] = '$UNCHANGED$') → adapt.instance_based._tradaboost.TrAdaBoost[source]

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to predict.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for domain parameter in predict.

Returns

selfobject: The updated object.

set_score_request(*, domain: Union[bool, None, str] = '$UNCHANGED$', sample_weight: Union[bool, None, str] = '$UNCHANGED$') → adapt.instance_based._tradaboost.TrAdaBoost[source]

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for domain parameter in score.
sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for sample_weight parameter in score.

Returns

selfobject: The updated object.

unsupervised_score(Xs, Xt)[source]

Return unsupervised score.

The normalized discrepancy distance is computed between the reweighted/transformed source input data and the target input data.

Parameters

Xsarray: Source input data.
Xtarray: Source input data.

Returns

scorefloat: Unsupervised score.

Examples

Reproduction of the TrAdaBoost experiments