adapt.instance_based.TwoStageTrAdaBoostR2

class adapt.instance_based.TwoStageTrAdaBoostR2(estimator=None, Xt=None, yt=None, n_estimators=10, n_estimators_fs=10, cv=5, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]

Two Stage Transfer AdaBoost for Regression

TwoStageTrAdaBoostR2 algorithm is a supervised instances-based domain adaptation method suited for regression tasks.

The method is based on a “reverse boosting” principle where the weights of source instances poorly predicted decrease at each boosting iteration whereas the ones of target instances increase.

This “two stages” version of TrAdaBoostR2 algorithm update separately the weights of source and target instances.

In a first stage, the weights of source instances are frozen whereas the ones of target instances are updated according to the classical AdaBoostR2 algorithm. In a second stage, the weights of target instances are now frozen whereas the ones of source instances are updated according to the TrAdaBoost algorithm.

At each first stage, a cross-validation score is computed with the labeled target data available. The CV scores obtained are used at the end to select the best estimator whithin all boosting iterations.

The algorithm performs the following steps:

  • 1. Normalize weights: \(\sum w_S + \sum w_T = 1\).

  • 2. Fit an AdaBoostR2 estimator \(f\) on source and target labeled data \((X_S, y_S), (X_T, y_T)\) with the respective importances initial weights: \(w_S, w_T\). During training of the AdaBoost estimator, the source weights \(w_S\) are frozen.

  • 3. Compute a cross-validation score on \((X_T, y_T)\)

  • 4. Compute error vectors of training instances:

    • \(\epsilon_S = L(f(X_S), y_S)\).

    • \(\epsilon_T = L(f(X_T), y_T)\).

  • 5 Normalize error vectors:

    • \(\epsilon_S = \epsilon_S \setminus max_{\epsilon \in \epsilon_S \cup \epsilon_T} \epsilon\).

    • \(\epsilon_T = \epsilon_T \setminus max_{\epsilon \in \epsilon_S \cup \epsilon_T} \epsilon\).

  • 6. Update source and target weights:

    • \(w_S = w_S \beta_S^{\epsilon_S} \setminus Z\).

    • \(w_T = w_T \setminus Z\).

    Where:

    • \(Z\) is a normalizing constant.

    • \(\beta_S\) is chosen such that the sum of target weights \(w_T\) is equal to \(\frac{n_T}{n_T + n_S} + \frac{t}{N - 1}(1 - \frac{n_T}{n_T + n_S})\) with \(t\) the current boosting iteration number. \(\beta_S\) is found using binary search.

  • 7. Return to step 1 and loop until the number \(N\) of boosting iteration is reached.

The prediction are then given by the best estimator according to cross-validation scores.

Parameters
estimatorsklearn estimator or tensorflow Model (default=None)

Estimator used to learn the task. If estimator is None, a LinearRegression instance is used as estimator.

Xtnumpy array (default=None)

Target input data.

ytnumpy array (default=None)

Target output data.

n_estimatorsint (default=10)

Number of boosting iteration.

n_estimators_fsint (default=10)

Number of boosting iteration in first stage (given to AdaboostR2 estimators)

cv: int, optional (default=5)

Split cross-validation parameter.

lrfloat (default=1.)

Learning rate. For higher lr, the sample weights are updating faster.

copyboolean (default=True)

Whether to make a copy of estimator or not.

verboseint (default=1)

Verbosity level.

random_stateint (default=None)

Seed of random generator.

paramskey, value arguments

Arguments given at the different level of the adapt object. It can be, for instance, compile or fit parameters of the estimator or kernel parameters etc… Accepted parameters can be found by calling the method _get_legal_params(params).

See also

TrAdaBoost, TrAdaBoostR2

References

1

[1] D. Pardoe and P. Stone. “Boosting for regression transfer”. In ICML, 2010.

Examples

>>> from sklearn.linear_model import Ridge
>>> from adapt.utils import make_regression_da
>>> from adapt.instance_based import TwoStageTrAdaBoostR2
>>> Xs, ys, Xt, yt = make_regression_da()
>>> model = TwoStageTrAdaBoostR2(Ridge(), n_estimators=10, Xt=Xt[:10], yt=yt[:10], random_state=0)
>>> model.fit(Xs, ys)
Iteration 0 - Cross-validation score: 0.2956 (0.0905)
Iteration 1 - Cross-validation score: 0.2956 (0.0905)
Iteration 2 - Cross-validation score: 0.2614 (0.1472)
Iteration 3 - Cross-validation score: 0.2701 (0.1362)
Iteration 4 - Cross-validation score: 0.2745 (0.1280)
Iteration 5 - Cross-validation score: 0.2768 (0.1228)
Iteration 6 - Cross-validation score: 0.2782 (0.1195)
Iteration 7 - Cross-validation score: 0.2783 (0.1170)
Iteration 8 - Cross-validation score: 0.2767 (0.1156)
Iteration 9 - Cross-validation score: 0.2702 (0.1155)
>>> model.score(Xt, yt)
0.5997110100905185
Attributes
estimators_list of object

List of fitted AdaboostR2 estimators for each first stage.

estimator_errors_1D array of float

Array of cross-validation MAE computed on labeled target data.

sample_weights_src_list of numpy arrays

List of source sample weight for each iteration.

sample_weights_tgt_list of numpy arrays

List of target sample weight for each iteration.

Methods

__init__([estimator, Xt, yt, n_estimators, ...])

fit(X, y[, Xt, yt, sample_weight_src, ...])

Fit TrAdaBoost

fit_estimator(X, y[, sample_weight, ...])

Fit estimator on X, y.

get_metadata_routing()

Get metadata routing of this object.

get_params([deep])

Get parameters for this estimator.

predict(X)

Return predictions of the best estimator according to cross-validation scores.

predict_estimator(X, **predict_params)

Return estimator predictions for X.

predict_weights([domain])

Return sample weights.

score(X, y)

Return the TrAdaboost score on X, y.

set_fit_request(*[, sample_weight_src, ...])

Request metadata passed to the fit method.

set_params(**params)

Set the parameters of this estimator.

set_predict_request(*[, domain])

Request metadata passed to the predict method.

set_score_request(*[, domain, sample_weight])

Request metadata passed to the score method.

unsupervised_score(Xs, Xt)

Return unsupervised score.
__init__(estimator=None, Xt=None, yt=None, n_estimators=10, n_estimators_fs=10, cv=5, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]
fit(X, y, Xt=None, yt=None, sample_weight_src=None, sample_weight_tgt=None, **fit_params)[source]

Fit TrAdaBoost

Parameters
Xnumpy array

Source input data.

ynumpy array

Source output data.

Xtarray (default=None)

Target input data. If None, the Xt argument given in init is used.

ytarray (default=None)

Target input data. If None, the Xt argument given in init is used.

sample_weight_srcnumpy array, (default=None)

Initial sample weight of source data

sample_weight_tgtnumpy array, (default=None)

Initial sample weight of target data

fit_paramskey, value arguments

Arguments given to the fit method of the estimator.

Returns
selfreturns an instance of self
fit_estimator(X, y, sample_weight=None, random_state=None, warm_start=True, **fit_params)[source]

Fit estimator on X, y.

Parameters
Xarray

Input data.

yarray

Output data.

sample_weightarray

Importance weighting.

random_stateint (default=None)

Seed of the random generator

warm_startbool (default=True)

If True, continue to fit estimator_, else, a new estimator is fitted based on a copy of estimator. (Be sure to set copy=True to use warm_start=False)

fit_paramskey, value arguments

Arguments given to the fit method of the estimator and to the compile method for tensorflow estimator.

Returns
estimator_fitted estimator
get_metadata_routing()[source]

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns
routingMetadataRequest

A MetadataRequest encapsulating routing information.

get_params(deep=True)[source]

Get parameters for this estimator.

Parameters
deepbool, default=True

Not used, here for scikit-learn compatibility.

Returns
paramsdict

Parameter names mapped to their values.

predict(X)[source]

Return predictions of the best estimator according to cross-validation scores.

Parameters
Xarray

Input data.

Returns
y_predarray

Best estimator predictions.

predict_estimator(X, **predict_params)[source]

Return estimator predictions for X.

Parameters
Xarray

input data

Returns
y_predarray

prediction of estimator.

predict_weights(domain='src')[source]

Return sample weights.

Return the source importance weighting of the best estimator.

You can secify between “source” and “target” weights with the domain parameter.

Parameters
domainstr (default=”tgt”)

Choose between "source", "src" and "target", "tgt".

Returns
weightssource sample weights
score(X, y)[source]

Return the TrAdaboost score on X, y.

Parameters
Xarray

input data

yarray

output data

Returns
scorefloat

estimator score.

set_fit_request(*, sample_weight_src: Union[bool, None, str] = '$UNCHANGED$', sample_weight_tgt: Union[bool, None, str] = '$UNCHANGED$') → adapt.instance_based._tradaboost.TwoStageTrAdaBoostR2[source]

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters
sample_weight_srcstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight_src parameter in fit.

sample_weight_tgtstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight_tgt parameter in fit.

Returns
selfobject

The updated object.

set_params(**params)[source]

Set the parameters of this estimator.

Parameters
**paramsdict

Estimator parameters.

Returns
selfestimator instance

Estimator instance.

set_predict_request(*, domain: Union[bool, None, str] = '$UNCHANGED$') → adapt.instance_based._tradaboost.TwoStageTrAdaBoostR2[source]

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters
domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for domain parameter in predict.

Returns
selfobject

The updated object.

set_score_request(*, domain: Union[bool, None, str] = '$UNCHANGED$', sample_weight: Union[bool, None, str] = '$UNCHANGED$') → adapt.instance_based._tradaboost.TwoStageTrAdaBoostR2[source]

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters
domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for domain parameter in score.

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns
selfobject

The updated object.

unsupervised_score(Xs, Xt)[source]

Return unsupervised score.

The normalized discrepancy distance is computed between the reweighted/transformed source input data and the target input data.

Parameters
Xsarray

Source input data.

Xtarray

Source input data.

Returns
scorefloat

Unsupervised score.

Examples