With the advancement of intelligent human-computer interaction technologies, UI control and scene recognition are playing an increasingly important role in fields such as automated testing and intelligent assisted operations. However, the issue of imbalanced few-shot samples poses challenges to recognition accuracy and generalization capabilities. To address this, this study proposes an efficient UI control and scene recognition method based on imbalanced few-shot samples, constructing a recognition system pipeline comprising basic core services and auxiliary functions. The core services include scene classification and general control detection, while the auxiliary functions combine few-shot transfer learning and click prior information to enhance the recognition of key controls. This study innovatively introduces a few-shot transfer learning strategy to improve model adaptability under data imbalance. The experimental results show that this method achieves a precision and recall rate of over 0.95 in scene classification tasks, and an mAP@0.5 (mean average precision) of above 0.8 in control detection, effectively improving the robustness and application value of UI recognition. This provides strong support for the development of automated and intelligent smart interaction systems.