Optimizing AI-Driven Resource Management: Hierarchical LDA, Autoencoders, and Iso map for Enhanced Dimensionality Reduction

Abstract

Artificial intelligence is changing the resource management of large-scale computing yet conventional allocation strategies find it challenging in dealing with high-dimensional data. Scalability and responsiveness suffer with high dimensional data. Advanced techniques in dimensionality reduction such as hierarchical latent dirichlet allocation autoencoders and iso map improve AI-based resource management, especially decision making and optimizing the efficiency of the computation. Scalability and responsiveness in cloud computing and big data environments can be enhanced using such AI-based techniques. In this study, a novel AI-based resource
management model has been proposed to enhance the accuracy, efficiency, and real-time allocation of resources. Topic-based feature extraction through hierarchical LDA, nonlinear compression using autoencoders, and geodesic distances are preserved by the iso map while reducing data complexity but retaining essential features. Results: The experiment has been done, and accuracy of 91.7% efficiency of 91.3%, and the computational cost reduction was found to be 78.3% that surpasses traditional approaches. Scalability and adaptability were enhanced in a high- performance computing environment. Reinforcement learning based optimization and Cloud edge AI are recommended to enhance the decision making for multi-cloud resource management.