Paper Link: Model Adaptation: Unsupervised Domain Adaptation Without Source Data (thecvf.com)

Key Elements

1. Target Generation:

   • Collaborative Class Conditional GAN (3CGAN)

     • Domain Discriminator D:

       $ \max_{\theta_D} \mathbb E_{x_t \sim D_t } [log D(x_t)] + \mathbb E_{y,z}[log(1-D(G(y,z)))]$

       Which is try to tell whether its inputs are original target samples or generated target sample

     • Generator G:

       $l_{adv}(G)=\mathbb E_{y,z}[logD(1-G(y,z))]$ $\min_{\theta_G}l_{adv}+\lambda_sl_{sem}$

       Which is trying to confuse the Domain Discriminator D

       And minimize the cross-entropy loss

     • Fixed Predictor C (Pretrained on source domain):

       $l_{sem}(G)=\mathbb E_{y,z}[-ylog_{p_{\theta_C}}(G(y,z))]$

       Which is there to check whether the generated samples are identical to its original labels

2. Model Adaptation:

   • This time we don’t fix the pretrained predictor C anymore

   • The final optimization objective:

     $\min_{\theta_C} \lambda_gl_{gen} + \lambda_w l_{wReg} + \lambda_{clu} l_{cluReg} ​$

   • Basic generation loss (which is the cross entropy loss for the generated target-like samples)

     $l_{gen}=\mathbb E_{y,z}[-ylog_{p_{\theta_C}}(x_g)]$

   • Weight Regularization (which is trying to make the newly trained classifier not too different from the original predictor C)

     $l_{wReg} =||\theta_C - \theta_{C_s}||^2$

   • Clustering-based Regularization (Entropy Minimization + KL Divergence between the distribution of target samples and the distribution of perturbated target samples, the KL Divergence is trying to make the prediction model locally smooth for each unlabeled target sample)

     $l_{cluReg} =\mathbb E_{x_t \sim D_t} [-p_{\theta_C}(x_t) log_{p_{\theta_C}}] + [KL(p_{\theta_C} (x_t) || p_{\theta_C} (x_t + \tilde r))]$

Zenvi

One Just Vegetable Chick

Comments

  Write a comment ...