adapt.instance_based.TrAdaBoostR2

class adapt.instance_based.TrAdaBoostR2(estimator=None, Xt=None, yt=None, n_estimators=10, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]

Transfer AdaBoost for Regression

TrAdaBoostR2 algorithm is a supervised instances-based domain adaptation method suited for regression tasks.

The method is based on a “reverse boosting” principle where the weights of source instances poorly predicted decrease at each boosting iteration whereas the ones of target instances increase.

The algorithm performs the following steps:

  • 1. Normalize weights: \(\sum w_S + \sum w_T = 1\).

  • 2. Fit an estimator \(f\) on source and target labeled data \((X_S, y_S), (X_T, y_T)\) with the respective importances weights: \(w_S, w_T\).

  • 3. Compute error vectors of training instances:

    • \(\epsilon_S = L(f(X_S), y_S)\).

    • \(\epsilon_T = L(f(X_T), y_T)\).

  • 4 Normalize error vectors:

    • \(\epsilon_S = \epsilon_S \setminus max_{\epsilon \in \epsilon_S \cup \epsilon_T} \epsilon\).

    • \(\epsilon_T = \epsilon_T \setminus max_{\epsilon \in \epsilon_S \cup \epsilon_T} \epsilon\).

  • 5. Compute total weighted error of target instances: \(E_T = \frac{1}{n_T} w_T^T \epsilon_T\).

  • 6. Update source and target weights:

    • \(w_S = w_S \beta^{\epsilon_S}\).

    • \(w_T = w_T \beta_T^{-\epsilon_T}\).

    Where:

    • \(\beta = 1 \setminus (1 + \sqrt{2 \text{ln} n_S \setminus N})\).

    • \(\beta_T = E_T \setminus (1 - E_T)\).

  • 7. Return to step 1 and loop until the number \(N\) of boosting iteration is reached.

The prediction are then given by the weighted median of the \(N \setminus 2\) last estimators.

Parameters
estimatorsklearn estimator or tensorflow Model (default=None)

Estimator used to learn the task. If estimator is None, a LinearRegression instance is used as estimator.

Xtnumpy array (default=None)

Target input data.

ytnumpy array (default=None)

Target output data.

n_estimatorsint (default=10)

Number of boosting iteration.

lrfloat (default=1.)

Learning rate. For higher lr, the sample weights are updating faster.

copyboolean (default=True)

Whether to make a copy of estimator or not.

verboseint (default=1)

Verbosity level.

random_stateint (default=None)

Seed of random generator.

paramskey, value arguments

Arguments given at the different level of the adapt object. It can be, for instance, compile or fit parameters of the estimator or kernel parameters etc… Accepted parameters can be found by calling the method _get_legal_params(params).

References

1

[1] D. Pardoe and P. Stone. “Boosting for regression transfer”. In ICML, 2010.

Examples

>>> from sklearn.linear_model import Ridge
>>> from adapt.utils import make_regression_da
>>> from adapt.instance_based import TrAdaBoostR2
>>> Xs, ys, Xt, yt = make_regression_da()
>>> model = TrAdaBoostR2(Ridge(), n_estimators=10, Xt=Xt[:10], yt=yt[:10], random_state=0)
>>> model.fit(Xs, ys)
Iteration 0 - Error: 0.4862
Iteration 1 - Error: 0.5711
Iteration 2 - Error: 0.6709
Iteration 3 - Error: 0.7095
Iteration 4 - Error: 0.7154
Iteration 5 - Error: 0.6987
Iteration 6 - Error: 0.6589
Iteration 7 - Error: 0.5907
Iteration 8 - Error: 0.4930
Iteration 9 - Error: 0.3666
>>> model.score(Xt, yt)
0.6998064452649377
Attributes
estimators_list of object

List of fitted estimators

estimator_errors_1D array of float

Array of weighted estimator errors computed on labeled target data.

estimator_weights_1D array of float

Array of estimator importance weights.

sample_weights_src_list of numpy arrays

List of source sample weight for each iteration.

sample_weights_tgt_list of numpy arrays

List of target sample weight for each iteration.

Methods

__init__([estimator, Xt, yt, n_estimators, ...])

fit(X, y[, Xt, yt, sample_weight_src, ...])

Fit TrAdaBoost

fit_estimator(X, y[, sample_weight, ...])

Fit estimator on X, y.

get_metadata_routing()

Get metadata routing of this object.

get_params([deep])

Get parameters for this estimator.

predict(X)

Return weighted median of estimators.

predict_estimator(X, **predict_params)

Return estimator predictions for X.

predict_weights([domain])

Return sample weights.

score(X, y)

Return the TrAdaboost score on X, y.

set_fit_request(*[, sample_weight_src, ...])

Request metadata passed to the fit method.

set_params(**params)

Set the parameters of this estimator.

set_predict_request(*[, domain])

Request metadata passed to the predict method.

set_score_request(*[, domain, sample_weight])

Request metadata passed to the score method.

unsupervised_score(Xs, Xt)

Return unsupervised score.

__init__(estimator=None, Xt=None, yt=None, n_estimators=10, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]
fit(X, y, Xt=None, yt=None, sample_weight_src=None, sample_weight_tgt=None, **fit_params)[source]

Fit TrAdaBoost

Parameters
Xnumpy array

Source input data.

ynumpy array

Source output data.

Xtarray (default=None)

Target input data. If None, the Xt argument given in init is used.

ytarray (default=None)

Target input data. If None, the Xt argument given in init is used.

sample_weight_srcnumpy array, (default=None)

Initial sample weight of source data

sample_weight_tgtnumpy array, (default=None)

Initial sample weight of target data

fit_paramskey, value arguments

Arguments given to the fit method of the estimator.

Returns
selfreturns an instance of self
fit_estimator(X, y, sample_weight=None, random_state=None, warm_start=True, **fit_params)[source]

Fit estimator on X, y.

Parameters
Xarray

Input data.

yarray

Output data.

sample_weightarray

Importance weighting.

random_stateint (default=None)

Seed of the random generator

warm_startbool (default=True)

If True, continue to fit estimator_, else, a new estimator is fitted based on a copy of estimator. (Be sure to set copy=True to use warm_start=False)

fit_paramskey, value arguments

Arguments given to the fit method of the estimator and to the compile method for tensorflow estimator.

Returns
estimator_fitted estimator
get_metadata_routing()[source]

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns
routingMetadataRequest

A MetadataRequest encapsulating routing information.

get_params(deep=True)[source]

Get parameters for this estimator.

Parameters
deepbool, default=True

Not used, here for scikit-learn compatibility.

Returns
paramsdict

Parameter names mapped to their values.

predict(X)[source]

Return weighted median of estimators.

Parameters
Xarray

Input data.

Returns
y_predarray

Median results.

predict_estimator(X, **predict_params)[source]

Return estimator predictions for X.

Parameters
Xarray

input data

Returns
y_predarray

prediction of estimator.

predict_weights(domain='src')[source]

Return sample weights.

Return the final importance weighting.

You can secify between “source” and “target” weights with the domain parameter.

Parameters
domainstr (default=”tgt”)

Choose between "source", "src" and "target", "tgt".

Returns
weightssource sample weights
score(X, y)[source]

Return the TrAdaboost score on X, y.

Parameters
Xarray

input data

yarray

output data

Returns
scorefloat

estimator score.

set_fit_request(*, sample_weight_src: Union[bool, None, str] = '$UNCHANGED$', sample_weight_tgt: Union[bool, None, str] = '$UNCHANGED$') adapt.instance_based._tradaboost.TrAdaBoostR2[source]

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters
sample_weight_srcstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight_src parameter in fit.

sample_weight_tgtstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight_tgt parameter in fit.

Returns
selfobject

The updated object.

set_params(**params)[source]

Set the parameters of this estimator.

Parameters
**paramsdict

Estimator parameters.

Returns
selfestimator instance

Estimator instance.

set_predict_request(*, domain: Union[bool, None, str] = '$UNCHANGED$') adapt.instance_based._tradaboost.TrAdaBoostR2[source]

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters
domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for domain parameter in predict.

Returns
selfobject

The updated object.

set_score_request(*, domain: Union[bool, None, str] = '$UNCHANGED$', sample_weight: Union[bool, None, str] = '$UNCHANGED$') adapt.instance_based._tradaboost.TrAdaBoostR2[source]

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters
domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for domain parameter in score.

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns
selfobject

The updated object.

unsupervised_score(Xs, Xt)[source]

Return unsupervised score.

The normalized discrepancy distance is computed between the reweighted/transformed source input data and the target input data.

Parameters
Xsarray

Source input data.

Xtarray

Source input data.

Returns
scorefloat

Unsupervised score.

Examples