Jul 30 22:47:33.395060 ip-10-0-131-130 systemd[1]: Starting Kubernetes Kubelet... Jul 30 22:47:33.839164 ip-10-0-131-130 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 22:47:33.839164 ip-10-0-131-130 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 30 22:47:33.839164 ip-10-0-131-130 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 22:47:33.839164 ip-10-0-131-130 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 30 22:47:33.839164 ip-10-0-131-130 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 22:47:33.840321 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.840233 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 30 22:47:33.845057 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845041 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 22:47:33.845057 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845056 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 22:47:33.845057 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845059 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845065 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845070 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845074 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845077 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845080 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845082 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845085 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845088 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845091 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845093 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845096 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845098 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845101 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845103 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845106 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845108 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845111 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845114 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 22:47:33.845152 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845117 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845128 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845132 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845136 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845139 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845142 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845144 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845147 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845150 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845153 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845156 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845159 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845161 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845165 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845167 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845170 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845173 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845176 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845178 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 22:47:33.845596 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845181 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845184 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845187 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845189 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845191 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845194 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845197 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845200 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845202 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845205 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845207 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845210 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845212 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845214 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845217 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845221 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845223 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845226 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845229 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845231 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 22:47:33.846110 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845234 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845236 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845239 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845241 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845244 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845246 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845249 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845251 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845254 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845257 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845259 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845262 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845265 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845267 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845270 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845272 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845275 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845277 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845280 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845283 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 22:47:33.846603 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845286 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845288 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845291 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845294 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845296 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845299 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845301 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845305 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845308 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845311 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845313 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845316 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845318 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845321 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845323 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845326 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845330 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845333 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.845336 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 22:47:33.847108 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846044 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846050 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846053 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846057 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846060 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846063 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846066 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846069 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846071 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846074 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846076 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846079 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846082 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846085 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846087 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846090 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846093 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846096 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846098 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846101 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 22:47:33.847570 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846105 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846109 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846112 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846115 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846119 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846123 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846126 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846129 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846132 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846134 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846137 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846139 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846142 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846144 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846147 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846149 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846152 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846155 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846158 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 22:47:33.848082 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846160 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846163 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846166 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846168 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846171 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846173 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846176 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846178 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846181 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846183 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846187 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846189 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846192 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846195 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846198 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846200 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846203 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846205 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846208 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846210 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 22:47:33.848541 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846213 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846215 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846218 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846220 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846223 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846225 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846228 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846230 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846233 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846235 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846238 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846241 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846244 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846246 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846249 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846251 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846254 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846256 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846259 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846261 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 22:47:33.849150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846263 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846266 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846268 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846271 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846273 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846277 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846279 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846283 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846285 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846288 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846291 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846293 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846296 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846299 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846302 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846305 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846307 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846309 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846312 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 22:47:33.849645 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846315 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846681 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846686 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846689 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846692 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846695 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846698 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846701 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846703 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846706 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846709 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846711 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846714 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846716 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846719 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846721 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846724 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846726 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846729 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846746 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 22:47:33.850151 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846751 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846754 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846758 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846761 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846764 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846766 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846770 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846773 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846776 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846779 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846782 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846784 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846787 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846790 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846792 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846795 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846797 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846800 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846803 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 22:47:33.850636 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846806 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846809 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846813 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846816 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846819 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846821 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846823 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846826 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846829 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846831 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846834 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846836 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846839 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846842 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846844 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846846 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846849 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846851 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846857 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846859 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 22:47:33.851122 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846862 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846864 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846867 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846869 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846871 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846874 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846876 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846879 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846881 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846884 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846886 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846889 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846893 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846895 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846898 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846901 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846904 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846906 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846909 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 22:47:33.851664 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846914 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846918 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846920 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846923 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846926 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846928 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846931 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846935 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846937 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846939 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846942 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846946 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846948 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846951 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846954 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846956 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846959 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846961 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846963 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846966 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 22:47:33.852154 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846968 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.846971 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847041 2554 flags.go:64] FLAG: --address="0.0.0.0" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847048 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847054 2554 flags.go:64] FLAG: --anonymous-auth="true" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847059 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847064 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847067 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847071 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847076 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847079 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847084 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847088 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847091 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847094 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847097 2554 flags.go:64] FLAG: --cgroup-root="" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847100 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847103 2554 flags.go:64] FLAG: --client-ca-file="" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847106 2554 flags.go:64] FLAG: --cloud-provider="external" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847109 2554 flags.go:64] FLAG: --cluster-dns="[]" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847113 2554 flags.go:64] FLAG: --cluster-domain="" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847116 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847119 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847124 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 30 22:47:33.852643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847127 2554 flags.go:64] FLAG: --container-log-max-files="5" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847131 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847134 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847137 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847141 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847144 2554 flags.go:64] FLAG: --contention-profiling="false" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847147 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847150 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847153 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847156 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847160 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847163 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847166 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847169 2554 flags.go:64] FLAG: --enable-load-reader="false" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847172 2554 flags.go:64] FLAG: --enable-server="true" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847175 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847179 2554 flags.go:64] FLAG: --event-burst="100" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847182 2554 flags.go:64] FLAG: --event-qps="50" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847185 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847189 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847192 2554 flags.go:64] FLAG: --eviction-hard="" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847195 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847198 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847201 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847204 2554 flags.go:64] FLAG: --eviction-soft="" Jul 30 22:47:33.853271 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847207 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847210 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847212 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847215 2554 flags.go:64] FLAG: --experimental-mounter-path="" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847218 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847221 2554 flags.go:64] FLAG: --fail-swap-on="true" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847225 2554 flags.go:64] FLAG: --feature-gates="" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847229 2554 flags.go:64] FLAG: --file-check-frequency="20s" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847232 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847235 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847238 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847242 2554 flags.go:64] FLAG: --healthz-port="10248" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847245 2554 flags.go:64] FLAG: --help="false" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847248 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-131-130.ec2.internal" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847251 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847254 2554 flags.go:64] FLAG: --http-check-frequency="20s" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847257 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847260 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847263 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847266 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847269 2554 flags.go:64] FLAG: --image-service-endpoint="" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847271 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847274 2554 flags.go:64] FLAG: --kube-api-burst="100" Jul 30 22:47:33.853903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847278 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847281 2554 flags.go:64] FLAG: --kube-api-qps="50" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847284 2554 flags.go:64] FLAG: --kube-reserved="" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847287 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847290 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847293 2554 flags.go:64] FLAG: --kubelet-cgroups="" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847296 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847299 2554 flags.go:64] FLAG: --lock-file="" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847302 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847305 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847308 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847313 2554 flags.go:64] FLAG: --log-json-split-stream="false" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847316 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847319 2554 flags.go:64] FLAG: --log-text-split-stream="false" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847322 2554 flags.go:64] FLAG: --logging-format="text" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847326 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847329 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847332 2554 flags.go:64] FLAG: --manifest-url="" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847335 2554 flags.go:64] FLAG: --manifest-url-header="" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847340 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847342 2554 flags.go:64] FLAG: --max-open-files="1000000" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847346 2554 flags.go:64] FLAG: --max-pods="110" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847349 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847352 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847355 2554 flags.go:64] FLAG: --memory-manager-policy="None" Jul 30 22:47:33.854491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847358 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847361 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847364 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847367 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847374 2554 flags.go:64] FLAG: --node-status-max-images="50" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847377 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847380 2554 flags.go:64] FLAG: --oom-score-adj="-999" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847383 2554 flags.go:64] FLAG: --pod-cidr="" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847386 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847392 2554 flags.go:64] FLAG: --pod-manifest-path="" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847395 2554 flags.go:64] FLAG: --pod-max-pids="-1" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847399 2554 flags.go:64] FLAG: --pods-per-core="0" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847402 2554 flags.go:64] FLAG: --port="10250" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847405 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847408 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06d682fb46cb9b9a8" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847411 2554 flags.go:64] FLAG: --qos-reserved="" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847414 2554 flags.go:64] FLAG: --read-only-port="10255" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847417 2554 flags.go:64] FLAG: --register-node="true" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847420 2554 flags.go:64] FLAG: --register-with-taints="" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847424 2554 flags.go:64] FLAG: --registry-burst="10" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847427 2554 flags.go:64] FLAG: --registry-qps="5" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847429 2554 flags.go:64] FLAG: --reserved-cpus="" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847434 2554 flags.go:64] FLAG: --reserved-memory="" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847437 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847441 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 30 22:47:33.855117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847444 2554 flags.go:64] FLAG: --rotate-certificates="false" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847446 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847449 2554 flags.go:64] FLAG: --runonce="false" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847452 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847455 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847458 2554 flags.go:64] FLAG: --seccomp-default="false" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847461 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847464 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847468 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847471 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847474 2554 flags.go:64] FLAG: --storage-driver-password="root" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847477 2554 flags.go:64] FLAG: --storage-driver-secure="false" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847480 2554 flags.go:64] FLAG: --storage-driver-table="stats" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847483 2554 flags.go:64] FLAG: --storage-driver-user="root" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847486 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847489 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847496 2554 flags.go:64] FLAG: --system-cgroups="" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847499 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847505 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847507 2554 flags.go:64] FLAG: --tls-cert-file="" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847510 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847515 2554 flags.go:64] FLAG: --tls-min-version="" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847518 2554 flags.go:64] FLAG: --tls-private-key-file="" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847520 2554 flags.go:64] FLAG: --topology-manager-policy="none" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847523 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 30 22:47:33.855743 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847526 2554 flags.go:64] FLAG: --topology-manager-scope="container" Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847529 2554 flags.go:64] FLAG: --v="2" Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847533 2554 flags.go:64] FLAG: --version="false" Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847537 2554 flags.go:64] FLAG: --vmodule="" Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847542 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847545 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847657 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847660 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847663 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847666 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847669 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847672 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847675 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847677 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847680 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847682 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847685 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847688 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847690 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847693 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847695 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 22:47:33.856393 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847698 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847700 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847703 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847706 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847710 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847713 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847716 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847718 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847721 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847724 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847726 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847729 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847744 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847747 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847749 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847754 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847756 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847759 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847761 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847764 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 22:47:33.856927 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847767 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847769 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847772 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847774 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847777 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847781 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847784 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847787 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847790 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847793 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847796 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847799 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847801 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847804 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847806 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847809 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847813 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847816 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847819 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 22:47:33.857414 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847823 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847827 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847829 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847833 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847836 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847840 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847842 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847845 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847849 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847851 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847854 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847857 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847859 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847862 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847864 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847867 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847869 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847871 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847874 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 22:47:33.857897 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847877 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847880 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847883 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847885 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847888 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847890 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847893 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847895 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847898 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847900 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847905 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847908 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847910 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847913 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847915 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847918 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847921 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847923 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847926 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847929 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 22:47:33.858365 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847931 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847935 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847938 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847940 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847943 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.847945 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.847952 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.855421 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.855435 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.855460 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.855468 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855541 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855550 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855560 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 22:47:33.858859 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855563 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855567 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855570 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855573 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855576 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855579 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855582 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855588 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855591 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855600 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855604 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855608 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855613 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855618 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855623 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855627 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855631 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855634 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855636 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 22:47:33.859207 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855640 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855642 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855645 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855648 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855650 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855653 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855655 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855658 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855660 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855663 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855665 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855668 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855670 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855673 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855675 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855677 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855680 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855682 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855685 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 22:47:33.859685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855687 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855690 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855694 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855699 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855710 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855713 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855716 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855718 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855721 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855723 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855726 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855728 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855741 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855744 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855747 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855749 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855752 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855755 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855758 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855760 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855763 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 22:47:33.860156 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855766 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855769 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855773 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855777 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855781 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855785 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855790 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855792 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855795 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855797 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855800 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855803 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855805 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855807 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855810 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855813 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855825 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855828 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855832 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855834 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 22:47:33.860680 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855837 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855839 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855842 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855844 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855847 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855849 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855853 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855857 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855861 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855865 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855869 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855872 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855874 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855877 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855879 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855882 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 22:47:33.861212 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.855884 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.855890 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856085 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856094 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856099 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856103 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856108 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856111 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856113 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856116 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856119 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856121 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856124 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856133 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856136 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 22:47:33.861600 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856138 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856141 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856143 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856147 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856151 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856154 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856156 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856159 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856161 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856164 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856166 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856172 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856177 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856182 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856186 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856189 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856192 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856194 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856197 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 22:47:33.862088 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856199 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856202 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856204 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856207 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856210 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856212 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856215 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856217 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856219 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856222 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856225 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856227 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856235 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856238 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856241 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856243 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856245 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856248 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856251 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856255 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 22:47:33.862536 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856259 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856263 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856267 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856270 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856272 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856275 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856277 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856280 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856282 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856285 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856288 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856290 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856293 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856295 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856298 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856300 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856304 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856307 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856310 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 22:47:33.863041 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856313 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856315 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856318 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856321 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856324 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856326 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856337 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856342 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856346 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856349 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856351 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856354 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856357 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856359 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856361 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856364 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856366 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856369 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856371 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856374 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 22:47:33.863491 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856376 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856379 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856381 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856383 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856386 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856388 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856391 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:33.856394 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.856399 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.856527 2554 server.go:962] "Client rotation is on, will bootstrap in background" Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.860302 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.861926 2554 server.go:1019] "Starting client certificate rotation" Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.862035 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 22:47:33.863986 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.862076 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 22:47:33.865965 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.865918 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 22:47:33.867401 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.867380 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 22:47:33.877128 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.877113 2554 log.go:25] "Validated CRI v1 runtime API" Jul 30 22:47:33.879551 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.879535 2554 log.go:25] "Validated CRI v1 image API" Jul 30 22:47:33.880345 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.880322 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 30 22:47:33.884783 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.884762 2554 fs.go:135] Filesystem UUIDs: map[3b9d50d8-1f4c-433d-a128-4df41a1600db:/dev/nvme0n1p3 6124532d-19e4-4122-adfc-e7b0dd13d4ea:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 30 22:47:33.884864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.884782 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 30 22:47:33.891092 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.890949 2554 manager.go:216] Machine: {Timestamp:2026-07-30 22:47:33.888553594 +0000 UTC m=+0.361017596 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099695 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec244fdf8bb2d0234581669124e2dc23 SystemUUID:ec244fdf-8bb2-d023-4581-669124e2dc23 BootID:9e50b0bd-74ae-4557-b649-f8b166fa60c1 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6d:a5:4e:c3:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6d:a5:4e:c3:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:39:12:d5:6e:e0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 30 22:47:33.891092 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.891086 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 30 22:47:33.891190 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.891167 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 30 22:47:33.894644 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.894615 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 30 22:47:33.894815 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.894647 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-130.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 30 22:47:33.895518 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.895508 2554 topology_manager.go:138] "Creating topology manager with none policy" Jul 30 22:47:33.895552 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.895519 2554 container_manager_linux.go:309] "Creating device plugin manager" Jul 30 22:47:33.895552 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.895527 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 22:47:33.895552 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.895541 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 22:47:33.897363 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.897352 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 30 22:47:33.900381 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.900365 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 22:47:33.901723 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.901711 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 30 22:47:33.901848 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.901838 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 30 22:47:33.901892 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.901868 2554 kubelet.go:486] "Attempting to sync node with API server" Jul 30 22:47:33.901892 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.901879 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 30 22:47:33.901971 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.901893 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 30 22:47:33.901971 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.901903 2554 kubelet.go:398] "Adding apiserver pod source" Jul 30 22:47:33.901971 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.901911 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 30 22:47:33.902287 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.902276 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 22:47:33.902323 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.902314 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 22:47:33.904700 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.904680 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 30 22:47:33.908021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.907989 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 30 22:47:33.908254 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.908237 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 30 22:47:33.909940 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909917 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 30 22:47:33.909940 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909945 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909953 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909961 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909973 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909979 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909987 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909992 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.909999 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.910004 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.910012 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 30 22:47:33.910021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.910018 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 30 22:47:33.910918 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.910908 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 30 22:47:33.910918 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.910917 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 30 22:47:33.913765 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.913725 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-130.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 30 22:47:33.914259 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.914238 2554 server.go:1268] "Started kubelet" Jul 30 22:47:33.914471 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.914440 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 30 22:47:33.914471 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.914447 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-130.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 30 22:47:33.914613 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.914487 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 30 22:47:33.915129 ip-10-0-131-130 systemd[1]: Started Kubernetes Kubelet. Jul 30 22:47:33.916172 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.915236 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 30 22:47:33.916172 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.915310 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 30 22:47:33.916496 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.916484 2554 server.go:310] "Adding debug handlers to kubelet server" Jul 30 22:47:33.918470 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.917793 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 30 22:47:33.921475 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.921457 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 30 22:47:33.923063 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.921404 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-130.ec2.internal.18c732d1cb30b445 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-130.ec2.internal,UID:ip-10-0-131-130.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-130.ec2.internal,},FirstTimestamp:2026-07-30 22:47:33.914211397 +0000 UTC m=+0.386675399,LastTimestamp:2026-07-30 22:47:33.914211397 +0000 UTC m=+0.386675399,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-130.ec2.internal,}" Jul 30 22:47:33.924227 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.924207 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 30 22:47:33.924227 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.924218 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 30 22:47:33.924372 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.924282 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:33.924995 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.924947 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 30 22:47:33.924995 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.924966 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 30 22:47:33.925115 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.925072 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 30 22:47:33.925163 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.925119 2554 reconstruct.go:97] "Volume reconstruction finished" Jul 30 22:47:33.925163 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.925127 2554 reconciler.go:29] "Reconciler: start to sync state" Jul 30 22:47:33.926866 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.926848 2554 factory.go:55] Registering systemd factory Jul 30 22:47:33.926866 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.926869 2554 factory.go:223] Registration of the systemd container factory successfully Jul 30 22:47:33.927128 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.927114 2554 factory.go:153] Registering CRI-O factory Jul 30 22:47:33.927195 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.927132 2554 factory.go:223] Registration of the crio container factory successfully Jul 30 22:47:33.927246 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.927198 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 30 22:47:33.927246 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.927222 2554 factory.go:103] Registering Raw factory Jul 30 22:47:33.927246 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.927237 2554 manager.go:1196] Started watching for new ooms in manager Jul 30 22:47:33.927580 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.927567 2554 manager.go:349] Starting recovery of all containers Jul 30 22:47:33.929400 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.929363 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 30 22:47:33.929505 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.929407 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-130.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 30 22:47:33.938381 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.938354 2554 manager.go:354] Recovery completed Jul 30 22:47:33.944000 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.943981 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 22:47:33.945808 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.945780 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientMemory" Jul 30 22:47:33.945886 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.945820 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 22:47:33.945886 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.945831 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientPID" Jul 30 22:47:33.946405 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.946386 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 30 22:47:33.946405 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.946403 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 30 22:47:33.946489 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.946420 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 30 22:47:33.946489 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.946393 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 22:47:33.947221 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.947203 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientMemory" Jul 30 22:47:33.947309 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.947234 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 22:47:33.947309 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.947249 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientPID" Jul 30 22:47:33.947700 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.947683 2554 policy_none.go:49] "None policy: Start" Jul 30 22:47:33.947700 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.947702 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 30 22:47:33.947847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.947713 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 30 22:47:33.948014 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.947945 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-130.ec2.internal.18c732d1cd12d125 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-130.ec2.internal,UID:ip-10-0-131-130.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-130.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-130.ec2.internal,},FirstTimestamp:2026-07-30 22:47:33.945807141 +0000 UTC m=+0.418271143,LastTimestamp:2026-07-30 22:47:33.945807141 +0000 UTC m=+0.418271143,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-130.ec2.internal,}" Jul 30 22:47:33.948752 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.948723 2554 policy_none.go:47] "Start" Jul 30 22:47:33.953747 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.953718 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2c79c" Jul 30 22:47:33.958728 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.958659 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-130.ec2.internal.18c732d1cd131993 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-130.ec2.internal,UID:ip-10-0-131-130.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-130.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-130.ec2.internal,},FirstTimestamp:2026-07-30 22:47:33.945825683 +0000 UTC m=+0.418289686,LastTimestamp:2026-07-30 22:47:33.945825683 +0000 UTC m=+0.418289686,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-130.ec2.internal,}" Jul 30 22:47:33.962990 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:33.962975 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2c79c" Jul 30 22:47:33.968413 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:33.968332 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-130.ec2.internal.18c732d1cd133f15 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-130.ec2.internal,UID:ip-10-0-131-130.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-130.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-130.ec2.internal,},FirstTimestamp:2026-07-30 22:47:33.945835285 +0000 UTC m=+0.418299288,LastTimestamp:2026-07-30 22:47:33.945835285 +0000 UTC m=+0.418299288,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-130.ec2.internal,}" Jul 30 22:47:34.025447 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.025398 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.028261 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.028236 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 30 22:47:34.029517 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.029492 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 30 22:47:34.029610 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.029548 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 30 22:47:34.029610 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.029583 2554 kubelet.go:2448] "Starting kubelet main sync loop" Jul 30 22:47:34.029707 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.029622 2554 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 30 22:47:34.032168 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.032151 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 22:47:34.121462 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.121316 2554 manager.go:337] "Starting Device Plugin manager" Jul 30 22:47:34.121564 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.121484 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 30 22:47:34.121564 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.121496 2554 server.go:87] "Starting device plugin registration server" Jul 30 22:47:34.121726 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.121715 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 30 22:47:34.121811 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.121728 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 30 22:47:34.121874 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.121843 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 30 22:47:34.121922 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.121913 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 30 22:47:34.121971 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.121922 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 30 22:47:34.122462 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.122438 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 30 22:47:34.122533 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.122497 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.130154 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.130124 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal"] Jul 30 22:47:34.130252 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.130204 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 22:47:34.131010 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.130994 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientMemory" Jul 30 22:47:34.131076 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.131023 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 22:47:34.131076 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.131036 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientPID" Jul 30 22:47:34.133098 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133084 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 22:47:34.133208 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133194 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.133257 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133222 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 22:47:34.133790 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133778 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientMemory" Jul 30 22:47:34.133867 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133814 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 22:47:34.133867 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133823 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientMemory" Jul 30 22:47:34.133867 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133842 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 22:47:34.133867 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133852 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientPID" Jul 30 22:47:34.133867 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.133826 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientPID" Jul 30 22:47:34.135751 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.135726 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.135801 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.135763 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 22:47:34.136753 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.136720 2554 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-130.ec2.internal\" not found" node="ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.137514 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.137491 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientMemory" Jul 30 22:47:34.137577 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.137523 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 22:47:34.137577 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.137532 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientPID" Jul 30 22:47:34.161441 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.161418 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-130.ec2.internal\" not found" node="ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.165821 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.165802 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-130.ec2.internal\" not found" node="ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.222341 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.222303 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 22:47:34.223299 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.223276 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientMemory" Jul 30 22:47:34.223396 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.223305 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 22:47:34.223396 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.223319 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientPID" Jul 30 22:47:34.223396 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.223340 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.226810 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.226793 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/232b7bf5bb7c90260e08f28534b57e67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal\" (UID: \"232b7bf5bb7c90260e08f28534b57e67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.226882 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.226817 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/232b7bf5bb7c90260e08f28534b57e67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal\" (UID: \"232b7bf5bb7c90260e08f28534b57e67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.226882 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.226840 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15e562856c5ed1d4d691baee256e1db7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-130.ec2.internal\" (UID: \"15e562856c5ed1d4d691baee256e1db7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.233982 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.233966 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.234036 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.233985 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-130.ec2.internal\": node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.266915 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.266886 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.327369 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.327326 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15e562856c5ed1d4d691baee256e1db7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-130.ec2.internal\" (UID: \"15e562856c5ed1d4d691baee256e1db7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.327369 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.327374 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/232b7bf5bb7c90260e08f28534b57e67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal\" (UID: \"232b7bf5bb7c90260e08f28534b57e67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.327565 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.327410 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/232b7bf5bb7c90260e08f28534b57e67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal\" (UID: \"232b7bf5bb7c90260e08f28534b57e67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.327565 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.327420 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15e562856c5ed1d4d691baee256e1db7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-130.ec2.internal\" (UID: \"15e562856c5ed1d4d691baee256e1db7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.327565 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.327466 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/232b7bf5bb7c90260e08f28534b57e67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal\" (UID: \"232b7bf5bb7c90260e08f28534b57e67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.327565 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.327501 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/232b7bf5bb7c90260e08f28534b57e67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal\" (UID: \"232b7bf5bb7c90260e08f28534b57e67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.367540 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.367489 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.464270 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.464229 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.468145 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.468128 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.468959 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.468937 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" Jul 30 22:47:34.568762 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.568714 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.669467 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.669399 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.771494 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.771462 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.862632 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.862603 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 30 22:47:34.863096 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.862787 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 22:47:34.872601 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.872583 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:34.919999 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.919931 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 22:47:34.969314 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.969276 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 22:42:34 +0000 UTC" deadline="2028-02-11 00:10:03.236256256 +0000 UTC" Jul 30 22:47:34.969314 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:34.969310 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13441h22m28.266949419s" Jul 30 22:47:34.973540 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:34.973515 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:35.074515 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:35.074482 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:35.095715 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:35.095684 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232b7bf5bb7c90260e08f28534b57e67.slice/crio-055fb3aac2c62f26ef14bb1c8e8053859ab9bc59cbc0c295042875da1774a63c WatchSource:0}: Error finding container 055fb3aac2c62f26ef14bb1c8e8053859ab9bc59cbc0c295042875da1774a63c: Status 404 returned error can't find the container with id 055fb3aac2c62f26ef14bb1c8e8053859ab9bc59cbc0c295042875da1774a63c Jul 30 22:47:35.097031 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:35.097000 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e562856c5ed1d4d691baee256e1db7.slice/crio-5dfc9189e21fdd8ddce27397adcd3e383fcad3032abef1c680dbcc7f91ea1b5a WatchSource:0}: Error finding container 5dfc9189e21fdd8ddce27397adcd3e383fcad3032abef1c680dbcc7f91ea1b5a: Status 404 returned error can't find the container with id 5dfc9189e21fdd8ddce27397adcd3e383fcad3032abef1c680dbcc7f91ea1b5a Jul 30 22:47:35.101198 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.101181 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 22:47:35.158651 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.158618 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 22:47:35.159754 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.159726 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientMemory" Jul 30 22:47:35.159836 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.159768 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 22:47:35.159836 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.159789 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeHasSufficientPID" Jul 30 22:47:35.175408 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:35.175353 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:35.276277 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:35.276240 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-130.ec2.internal\" not found" Jul 30 22:47:35.308409 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.308380 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 22:47:35.324585 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.324555 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" Jul 30 22:47:35.338788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.338764 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 22:47:35.340554 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.340540 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" Jul 30 22:47:35.354359 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.354341 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 22:47:35.499472 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.499385 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 22:47:35.903910 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.903823 2554 apiserver.go:52] "Watching apiserver" Jul 30 22:47:35.911184 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.911154 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 30 22:47:35.911675 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.911641 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-mws57","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz","openshift-cluster-node-tuning-operator/tuned-jg22k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal","openshift-multus/multus-additional-cni-plugins-bfnwj","openshift-multus/network-metrics-daemon-2n7px","openshift-network-operator/iptables-alerter-hh7d2","kube-system/global-pull-secret-syncer-spp8w","kube-system/konnectivity-agent-t9f9p","kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal","openshift-dns/node-resolver-mm57w","openshift-image-registry/node-ca-9srwj","openshift-multus/multus-jqkvm","openshift-network-diagnostics/network-check-target-z72p4"] Jul 30 22:47:35.914487 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.914302 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.916787 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.916726 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:35.917307 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.917281 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hzw64\"" Jul 30 22:47:35.917307 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.917300 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 30 22:47:35.917545 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.917340 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 30 22:47:35.917648 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.917627 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 30 22:47:35.917648 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.917642 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 30 22:47:35.918991 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.918966 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.920936 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.920883 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 30 22:47:35.921536 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.921210 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 30 22:47:35.921675 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.921548 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.922508 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.922127 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s45r8\"" Jul 30 22:47:35.922508 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.922231 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 30 22:47:35.922508 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.922446 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jdjh7\"" Jul 30 22:47:35.922876 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.922854 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 30 22:47:35.922964 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.922949 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 30 22:47:35.924329 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.924299 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.924399 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.924378 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 30 22:47:35.924587 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.924568 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 30 22:47:35.926061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.925513 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 30 22:47:35.926061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.925758 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 30 22:47:35.926061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.925846 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 30 22:47:35.926061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.925871 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v9266\"" Jul 30 22:47:35.926061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.925976 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 30 22:47:35.926061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.925987 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 30 22:47:35.926782 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.926759 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g2hsb\"" Jul 30 22:47:35.926879 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.926807 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 30 22:47:35.927044 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.927026 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 30 22:47:35.928076 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.928058 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:35.928185 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:35.928131 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:35.930501 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.930479 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:35.932998 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.932978 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:35.933113 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:35.933047 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:35.938890 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.938233 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 30 22:47:35.938890 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.938634 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 30 22:47:35.938890 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.938689 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 30 22:47:35.939095 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939047 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bmgmq\"" Jul 30 22:47:35.939095 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939073 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:35.939221 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939129 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqmvb\" (UniqueName: \"kubernetes.io/projected/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-kube-api-access-bqmvb\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:35.939221 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939178 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-cni-multus\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.939326 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939219 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsv4n\" (UniqueName: \"kubernetes.io/projected/dfd556d1-cf73-4e72-9959-988c7d67484e-kube-api-access-fsv4n\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.939326 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939278 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-lib-modules\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.939326 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939319 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-log-socket\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.939478 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939357 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.939478 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939443 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-socket-dir-parent\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.939613 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939480 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:35.939613 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939518 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-device-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:35.939613 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939568 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-cni-bin\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.939788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939602 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-cni-netd\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.939788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939649 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.939788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939764 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-var-lib-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.939981 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939860 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfd556d1-cf73-4e72-9959-988c7d67484e-cni-binary-copy\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.939981 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939913 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-daemon-config\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.939981 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939962 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-kubernetes\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.940139 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939997 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysctl-conf\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.940139 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.939920 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:35.940139 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940078 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae00cd60-e89a-4db6-af10-7004eba7c1d3-tmp\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.940296 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940139 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-var-lib-kubelet\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.940296 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940184 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-etc-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.940296 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940225 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-cni-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.940296 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:35.940217 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:35.940549 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940364 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovnkube-script-lib\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.940549 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940403 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2fsp\" (UniqueName: \"kubernetes.io/projected/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-kube-api-access-n2fsp\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.940549 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940429 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-modprobe-d\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.940549 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940474 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-conf-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.940549 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940506 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-sys\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.940869 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940680 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-systemd-units\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.940869 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940751 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-env-overrides\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.940869 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940794 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-system-cni-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.940869 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940827 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-netns\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.940869 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940859 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-socket-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:35.941164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940922 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-os-release\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.941164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940954 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-slash\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.941164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.940982 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovnkube-config\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.941164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941026 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovn-node-metrics-cert\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.941164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-system-cni-dir\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.941164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941092 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-k8s-cni-cncf-io\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.941164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941120 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-kubelet\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.941164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941150 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-host\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.941639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941217 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-node-log\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.941639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941332 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24vjd\" (UniqueName: \"kubernetes.io/projected/ca438473-69cd-4a33-89de-580b19abe276-kube-api-access-24vjd\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:35.941639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941397 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-cni-bin\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.941639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941477 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 22:47:35.941639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941509 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-hostroot\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.941639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941580 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-etc-kubernetes\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.941639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941615 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysconfig\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.942028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941740 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-tuned\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.942028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941794 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnzx8\" (UniqueName: \"kubernetes.io/projected/ae00cd60-e89a-4db6-af10-7004eba7c1d3-kube-api-access-nnzx8\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.942028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941896 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:35.942028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941929 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-cnibin\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.942028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.941988 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-multus-certs\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.942028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942000 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 30 22:47:35.942293 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942024 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-registration-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:35.942293 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942094 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-run\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.942293 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942139 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-run-netns\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.942293 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942153 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 30 22:47:35.942293 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942175 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-ovn\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.942293 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942219 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-sys-fs\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:35.942612 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942292 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-etc-selinux\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:35.942612 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942428 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-systemd\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.942612 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942464 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.942612 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942535 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.942864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942612 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.942864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942688 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.942864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942734 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:35.942864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942723 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.942864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942794 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-657mt\" (UniqueName: \"kubernetes.io/projected/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-kube-api-access-657mt\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.942864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942828 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-os-release\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:35.943195 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942867 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysctl-d\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.943195 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942896 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-systemd\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:35.943195 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942929 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-kubelet\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:35.943195 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.942962 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cnibin\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:35.944264 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.944246 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r2khm\"" Jul 30 22:47:35.945806 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.945784 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 30 22:47:35.945928 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.945901 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v6fg8\"" Jul 30 22:47:35.947094 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.946809 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:35.947282 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.947263 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 30 22:47:35.950026 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.949867 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 30 22:47:35.950026 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.949901 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qwbkl\"" Jul 30 22:47:35.950026 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.949911 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 30 22:47:35.950026 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.949919 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 30 22:47:35.972112 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.972091 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r9zxf" Jul 30 22:47:35.984894 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:35.984871 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r9zxf" Jul 30 22:47:36.026904 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.026881 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 30 22:47:36.036886 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.036816 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" event={"ID":"15e562856c5ed1d4d691baee256e1db7","Type":"ContainerStarted","Data":"5dfc9189e21fdd8ddce27397adcd3e383fcad3032abef1c680dbcc7f91ea1b5a"} Jul 30 22:47:36.038346 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.038314 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" event={"ID":"232b7bf5bb7c90260e08f28534b57e67","Type":"ContainerStarted","Data":"055fb3aac2c62f26ef14bb1c8e8053859ab9bc59cbc0c295042875da1774a63c"} Jul 30 22:47:36.044037 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044014 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:36.044172 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044051 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-cnibin\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.044172 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044080 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-multus-certs\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.044280 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.044229 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:36.044330 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044296 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-registration-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.044386 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044351 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-run\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.044440 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044380 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-run-netns\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.044440 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044382 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-multus-certs\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.044440 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044419 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-ovn\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.044597 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044450 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-dbus\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.044597 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044483 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-sys-fs\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.044597 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.044520 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs podName:25f75d33-3b48-4851-b5d8-5e7cd1db81ed nodeName:}" failed. No retries permitted until 2026-07-30 22:47:36.544484278 +0000 UTC m=+3.016948269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs") pod "network-metrics-daemon-2n7px" (UID: "25f75d33-3b48-4851-b5d8-5e7cd1db81ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:36.044597 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044521 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-cnibin\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.044597 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044549 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-etc-selinux\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.044597 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044571 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-systemd\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.044597 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044581 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-ovn\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.044597 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044587 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-registration-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044592 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044554 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-sys-fs\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044619 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044657 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-run-netns\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044666 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044699 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044711 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-run\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044725 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044742 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-run-systemd\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044743 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044776 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-657mt\" (UniqueName: \"kubernetes.io/projected/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-kube-api-access-657mt\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044801 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-etc-selinux\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044842 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044844 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044892 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cvfz\" (UniqueName: \"kubernetes.io/projected/e2668b41-384e-433b-8487-35040f19364f-kube-api-access-6cvfz\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044927 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-os-release\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.045036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044967 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysctl-d\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.044993 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-systemd\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045018 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-kubelet\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045042 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cnibin\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045096 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqmvb\" (UniqueName: \"kubernetes.io/projected/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-kube-api-access-bqmvb\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045140 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-cni-multus\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045159 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsv4n\" (UniqueName: \"kubernetes.io/projected/dfd556d1-cf73-4e72-9959-988c7d67484e-kube-api-access-fsv4n\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045160 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cnibin\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045177 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-lib-modules\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045305 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-cni-multus\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045420 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-os-release\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045440 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-log-socket\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045482 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045509 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-lib-modules\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045494 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-systemd\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045560 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-log-socket\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045571 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysctl-d\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.045881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045572 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-socket-dir-parent\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045622 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-socket-dir-parent\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045620 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045659 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-device-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045673 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045682 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-cni-bin\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-cni-netd\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045736 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-cni-bin\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045728 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-device-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045734 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045814 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-cni-netd\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045818 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045848 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-var-lib-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045864 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-kubelet\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045871 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045873 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfd556d1-cf73-4e72-9959-988c7d67484e-cni-binary-copy\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045915 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.046847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045922 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-daemon-config\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045948 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-kubernetes\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045956 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-var-lib-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.045986 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046003 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-kubernetes\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046005 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysctl-conf\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046095 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysctl-conf\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046095 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae00cd60-e89a-4db6-af10-7004eba7c1d3-tmp\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046133 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-var-lib-kubelet\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046165 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-etc-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046173 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-var-lib-kubelet\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046197 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf94eccb-32e3-45b3-9329-8991e4f7d720-tmp-dir\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046219 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-etc-openvswitch\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046225 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-cni-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046252 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovnkube-script-lib\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046276 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fsp\" (UniqueName: \"kubernetes.io/projected/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-kube-api-access-n2fsp\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046298 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ab13381-4217-4fbd-a559-07f05259738f-konnectivity-ca\") pod \"konnectivity-agent-t9f9p\" (UID: \"2ab13381-4217-4fbd-a559-07f05259738f\") " pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046320 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f340f7c6-8994-43b9-a22a-8b7a4883518c-host\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.047679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046322 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046345 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2668b41-384e-433b-8487-35040f19364f-iptables-alerter-script\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046358 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-cni-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046373 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-modprobe-d\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046402 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-conf-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046424 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-sys\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046434 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-daemon-config\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046459 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfd556d1-cf73-4e72-9959-988c7d67484e-cni-binary-copy\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046530 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-multus-conf-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046547 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-sys\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046614 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-systemd-units\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046624 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-modprobe-d\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046644 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-env-overrides\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046668 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-systemd-units\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046674 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-system-cni-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046738 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-netns\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-socket-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.048681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046856 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-system-cni-dir\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046859 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-os-release\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046911 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-netns\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046954 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf94eccb-32e3-45b3-9329-8991e4f7d720-hosts-file\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046956 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-os-release\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.046980 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2668b41-384e-433b-8487-35040f19364f-host-slash\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047020 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-slash\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047046 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovnkube-config\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047083 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca438473-69cd-4a33-89de-580b19abe276-socket-dir\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047096 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovn-node-metrics-cert\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047126 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtzc9\" (UniqueName: \"kubernetes.io/projected/bf94eccb-32e3-45b3-9329-8991e4f7d720-kube-api-access-rtzc9\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047133 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-env-overrides\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047172 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svnzc\" (UniqueName: \"kubernetes.io/projected/f340f7c6-8994-43b9-a22a-8b7a4883518c-kube-api-access-svnzc\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047200 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-system-cni-dir\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047204 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-host-slash\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047228 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-kubelet-config\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047255 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ab13381-4217-4fbd-a559-07f05259738f-agent-certs\") pod \"konnectivity-agent-t9f9p\" (UID: \"2ab13381-4217-4fbd-a559-07f05259738f\") " pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:36.049660 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047266 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-system-cni-dir\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047282 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-k8s-cni-cncf-io\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047327 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-run-k8s-cni-cncf-io\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047363 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-kubelet\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047391 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-host\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047417 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-node-log\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047422 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovnkube-script-lib\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047467 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f340f7c6-8994-43b9-a22a-8b7a4883518c-serviceca\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047479 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-host\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047499 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24vjd\" (UniqueName: \"kubernetes.io/projected/ca438473-69cd-4a33-89de-580b19abe276-kube-api-access-24vjd\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047527 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-cni-bin\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047530 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-node-log\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047571 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-kubelet\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047572 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-host-var-lib-cni-bin\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047581 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovnkube-config\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047596 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-hostroot\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047625 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-etc-kubernetes\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047661 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-hostroot\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.050590 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047686 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysconfig\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.051579 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047719 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd556d1-cf73-4e72-9959-988c7d67484e-etc-kubernetes\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.051579 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047777 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-tuned\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.051579 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047794 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-sysconfig\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.051579 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047855 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnzx8\" (UniqueName: \"kubernetes.io/projected/ae00cd60-e89a-4db6-af10-7004eba7c1d3-kube-api-access-nnzx8\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.051579 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.047991 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.053439 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.053377 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-ovn-node-metrics-cert\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.054769 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.054412 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae00cd60-e89a-4db6-af10-7004eba7c1d3-etc-tuned\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.054769 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.054727 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae00cd60-e89a-4db6-af10-7004eba7c1d3-tmp\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.058297 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.058265 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsv4n\" (UniqueName: \"kubernetes.io/projected/dfd556d1-cf73-4e72-9959-988c7d67484e-kube-api-access-fsv4n\") pod \"multus-jqkvm\" (UID: \"dfd556d1-cf73-4e72-9959-988c7d67484e\") " pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.061107 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.061084 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnzx8\" (UniqueName: \"kubernetes.io/projected/ae00cd60-e89a-4db6-af10-7004eba7c1d3-kube-api-access-nnzx8\") pod \"tuned-jg22k\" (UID: \"ae00cd60-e89a-4db6-af10-7004eba7c1d3\") " pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.061653 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.061627 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqmvb\" (UniqueName: \"kubernetes.io/projected/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-kube-api-access-bqmvb\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:36.063487 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.063458 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24vjd\" (UniqueName: \"kubernetes.io/projected/ca438473-69cd-4a33-89de-580b19abe276-kube-api-access-24vjd\") pod \"aws-ebs-csi-driver-node-kf7nz\" (UID: \"ca438473-69cd-4a33-89de-580b19abe276\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.063611 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.063579 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2fsp\" (UniqueName: \"kubernetes.io/projected/ce1d4419-eb7f-4e08-8a7b-824a9baf39ee-kube-api-access-n2fsp\") pod \"ovnkube-node-mws57\" (UID: \"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.064563 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.064537 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-657mt\" (UniqueName: \"kubernetes.io/projected/9ff92d75-8091-4a72-a19b-6a39fef8f5d9-kube-api-access-657mt\") pod \"multus-additional-cni-plugins-bfnwj\" (UID: \"9ff92d75-8091-4a72-a19b-6a39fef8f5d9\") " pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.127970 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.127939 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 22:47:36.149136 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149103 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2668b41-384e-433b-8487-35040f19364f-host-slash\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.149275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149142 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtzc9\" (UniqueName: \"kubernetes.io/projected/bf94eccb-32e3-45b3-9329-8991e4f7d720-kube-api-access-rtzc9\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.149275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149167 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svnzc\" (UniqueName: \"kubernetes.io/projected/f340f7c6-8994-43b9-a22a-8b7a4883518c-kube-api-access-svnzc\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.149275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149190 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-kubelet-config\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.149275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149212 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ab13381-4217-4fbd-a559-07f05259738f-agent-certs\") pod \"konnectivity-agent-t9f9p\" (UID: \"2ab13381-4217-4fbd-a559-07f05259738f\") " pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:36.149275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149224 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2668b41-384e-433b-8487-35040f19364f-host-slash\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.149275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149237 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f340f7c6-8994-43b9-a22a-8b7a4883518c-serviceca\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149284 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-kubelet-config\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149301 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-dbus\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149338 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cvfz\" (UniqueName: \"kubernetes.io/projected/e2668b41-384e-433b-8487-35040f19364f-kube-api-access-6cvfz\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149441 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-dbus\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149475 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149519 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf94eccb-32e3-45b3-9329-8991e4f7d720-tmp-dir\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149547 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ab13381-4217-4fbd-a559-07f05259738f-konnectivity-ca\") pod \"konnectivity-agent-t9f9p\" (UID: \"2ab13381-4217-4fbd-a559-07f05259738f\") " pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149562 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f340f7c6-8994-43b9-a22a-8b7a4883518c-host\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149586 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2668b41-384e-433b-8487-35040f19364f-iptables-alerter-script\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.149620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149614 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.150053 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149634 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f340f7c6-8994-43b9-a22a-8b7a4883518c-host\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.150053 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149642 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf94eccb-32e3-45b3-9329-8991e4f7d720-hosts-file\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.150053 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149694 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf94eccb-32e3-45b3-9329-8991e4f7d720-hosts-file\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.150053 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.149751 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:36.150053 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149793 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f340f7c6-8994-43b9-a22a-8b7a4883518c-serviceca\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.150053 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.149830 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret podName:ff6d597e-b9e7-4074-b08e-0ab2234d4a3a nodeName:}" failed. No retries permitted until 2026-07-30 22:47:36.649811169 +0000 UTC m=+3.122275169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret") pod "global-pull-secret-syncer-spp8w" (UID: "ff6d597e-b9e7-4074-b08e-0ab2234d4a3a") : object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:36.150053 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.149898 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf94eccb-32e3-45b3-9329-8991e4f7d720-tmp-dir\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.150282 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.150192 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ab13381-4217-4fbd-a559-07f05259738f-konnectivity-ca\") pod \"konnectivity-agent-t9f9p\" (UID: \"2ab13381-4217-4fbd-a559-07f05259738f\") " pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:36.150282 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.150199 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2668b41-384e-433b-8487-35040f19364f-iptables-alerter-script\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.151677 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.151655 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ab13381-4217-4fbd-a559-07f05259738f-agent-certs\") pod \"konnectivity-agent-t9f9p\" (UID: \"2ab13381-4217-4fbd-a559-07f05259738f\") " pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:36.159169 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.159119 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 22:47:36.159169 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.159147 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 22:47:36.159169 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.159163 2554 projected.go:196] Error preparing data for projected volume kube-api-access-6whhx for pod openshift-network-diagnostics/network-check-target-z72p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:36.159320 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.159219 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx podName:a14e59aa-d437-481b-803f-1dacc98a952c nodeName:}" failed. No retries permitted until 2026-07-30 22:47:36.659204464 +0000 UTC m=+3.131668457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6whhx" (UniqueName: "kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx") pod "network-check-target-z72p4" (UID: "a14e59aa-d437-481b-803f-1dacc98a952c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:36.161486 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.161453 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cvfz\" (UniqueName: \"kubernetes.io/projected/e2668b41-384e-433b-8487-35040f19364f-kube-api-access-6cvfz\") pod \"iptables-alerter-hh7d2\" (UID: \"e2668b41-384e-433b-8487-35040f19364f\") " pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.161694 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.161676 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtzc9\" (UniqueName: \"kubernetes.io/projected/bf94eccb-32e3-45b3-9329-8991e4f7d720-kube-api-access-rtzc9\") pod \"node-resolver-mm57w\" (UID: \"bf94eccb-32e3-45b3-9329-8991e4f7d720\") " pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.161772 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.161711 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svnzc\" (UniqueName: \"kubernetes.io/projected/f340f7c6-8994-43b9-a22a-8b7a4883518c-kube-api-access-svnzc\") pod \"node-ca-9srwj\" (UID: \"f340f7c6-8994-43b9-a22a-8b7a4883518c\") " pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.229809 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.229775 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jqkvm" Jul 30 22:47:36.245323 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.245286 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" Jul 30 22:47:36.250454 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.250423 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jg22k" Jul 30 22:47:36.259393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.259370 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:47:36.266182 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.266156 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" Jul 30 22:47:36.275096 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.275072 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hh7d2" Jul 30 22:47:36.285943 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.285924 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:36.294672 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.294652 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mm57w" Jul 30 22:47:36.301632 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.301613 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9srwj" Jul 30 22:47:36.552984 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.552903 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:36.553153 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.553044 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:36.553153 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.553120 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs podName:25f75d33-3b48-4851-b5d8-5e7cd1db81ed nodeName:}" failed. No retries permitted until 2026-07-30 22:47:37.553099302 +0000 UTC m=+4.025563300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs") pod "network-metrics-daemon-2n7px" (UID: "25f75d33-3b48-4851-b5d8-5e7cd1db81ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:36.653636 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.653597 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:36.653843 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.653764 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:36.653843 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.653841 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret podName:ff6d597e-b9e7-4074-b08e-0ab2234d4a3a nodeName:}" failed. No retries permitted until 2026-07-30 22:47:37.653821761 +0000 UTC m=+4.126285776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret") pod "global-pull-secret-syncer-spp8w" (UID: "ff6d597e-b9e7-4074-b08e-0ab2234d4a3a") : object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:36.755105 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.755066 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:36.755274 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.755242 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 22:47:36.755274 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.755272 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 22:47:36.755361 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.755288 2554 projected.go:196] Error preparing data for projected volume kube-api-access-6whhx for pod openshift-network-diagnostics/network-check-target-z72p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:36.755361 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:36.755345 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx podName:a14e59aa-d437-481b-803f-1dacc98a952c nodeName:}" failed. No retries permitted until 2026-07-30 22:47:37.755330641 +0000 UTC m=+4.227794631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6whhx" (UniqueName: "kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx") pod "network-check-target-z72p4" (UID: "a14e59aa-d437-481b-803f-1dacc98a952c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:36.988856 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.988765 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 22:42:36 +0000 UTC" deadline="2028-01-06 07:29:04.978356951 +0000 UTC" Jul 30 22:47:36.988856 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:36.988811 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12584h41m27.989552755s" Jul 30 22:47:37.303029 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:37.302441 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd556d1_cf73_4e72_9959_988c7d67484e.slice/crio-4e45e5c6283035aea342aae9e0a233e44a57d4088fc6462fc4f1fcd8a678eaaa WatchSource:0}: Error finding container 4e45e5c6283035aea342aae9e0a233e44a57d4088fc6462fc4f1fcd8a678eaaa: Status 404 returned error can't find the container with id 4e45e5c6283035aea342aae9e0a233e44a57d4088fc6462fc4f1fcd8a678eaaa Jul 30 22:47:37.303427 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:37.303397 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf340f7c6_8994_43b9_a22a_8b7a4883518c.slice/crio-417dd9a85d452670361ee40b28a8a95dffe8b31f4f9f08fbc233737ed5c7f1f2 WatchSource:0}: Error finding container 417dd9a85d452670361ee40b28a8a95dffe8b31f4f9f08fbc233737ed5c7f1f2: Status 404 returned error can't find the container with id 417dd9a85d452670361ee40b28a8a95dffe8b31f4f9f08fbc233737ed5c7f1f2 Jul 30 22:47:37.304261 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:37.304165 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab13381_4217_4fbd_a559_07f05259738f.slice/crio-0132d1eeb933f82d40e8c0f1ecf8338b840854d74b8577c79ca09067b60d5773 WatchSource:0}: Error finding container 0132d1eeb933f82d40e8c0f1ecf8338b840854d74b8577c79ca09067b60d5773: Status 404 returned error can't find the container with id 0132d1eeb933f82d40e8c0f1ecf8338b840854d74b8577c79ca09067b60d5773 Jul 30 22:47:37.305319 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:37.305274 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1d4419_eb7f_4e08_8a7b_824a9baf39ee.slice/crio-a3dd2a47902d71adc07070662947f2ad3cf74ec69ce0ac09473cb46db40b0eed WatchSource:0}: Error finding container a3dd2a47902d71adc07070662947f2ad3cf74ec69ce0ac09473cb46db40b0eed: Status 404 returned error can't find the container with id a3dd2a47902d71adc07070662947f2ad3cf74ec69ce0ac09473cb46db40b0eed Jul 30 22:47:37.308489 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:37.308461 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2668b41_384e_433b_8487_35040f19364f.slice/crio-04a19891ba45b7d8afe16761d447e600a05384c37282323704e6b6d5b17a6e7f WatchSource:0}: Error finding container 04a19891ba45b7d8afe16761d447e600a05384c37282323704e6b6d5b17a6e7f: Status 404 returned error can't find the container with id 04a19891ba45b7d8afe16761d447e600a05384c37282323704e6b6d5b17a6e7f Jul 30 22:47:37.310720 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:37.309974 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae00cd60_e89a_4db6_af10_7004eba7c1d3.slice/crio-6cf6fe247fb9902b47e167f0a176d6c6d1c2b298b7cd855a70628f3ae19b43b9 WatchSource:0}: Error finding container 6cf6fe247fb9902b47e167f0a176d6c6d1c2b298b7cd855a70628f3ae19b43b9: Status 404 returned error can't find the container with id 6cf6fe247fb9902b47e167f0a176d6c6d1c2b298b7cd855a70628f3ae19b43b9 Jul 30 22:47:37.310940 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:37.310719 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf94eccb_32e3_45b3_9329_8991e4f7d720.slice/crio-a71f6a5637c54a7c01d6ba7c2b477cc660434995e2033da84cf8901a4f1f8ddc WatchSource:0}: Error finding container a71f6a5637c54a7c01d6ba7c2b477cc660434995e2033da84cf8901a4f1f8ddc: Status 404 returned error can't find the container with id a71f6a5637c54a7c01d6ba7c2b477cc660434995e2033da84cf8901a4f1f8ddc Jul 30 22:47:37.311698 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:47:37.311443 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff92d75_8091_4a72_a19b_6a39fef8f5d9.slice/crio-ef01e9041e8f37ff187c95b1fa87c24b21d1f84471674af33e92d06174dce61e WatchSource:0}: Error finding container ef01e9041e8f37ff187c95b1fa87c24b21d1f84471674af33e92d06174dce61e: Status 404 returned error can't find the container with id ef01e9041e8f37ff187c95b1fa87c24b21d1f84471674af33e92d06174dce61e Jul 30 22:47:37.564093 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:37.563902 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:37.564211 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:37.564050 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:37.564211 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:37.564205 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs podName:25f75d33-3b48-4851-b5d8-5e7cd1db81ed nodeName:}" failed. No retries permitted until 2026-07-30 22:47:39.564189903 +0000 UTC m=+6.036653894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs") pod "network-metrics-daemon-2n7px" (UID: "25f75d33-3b48-4851-b5d8-5e7cd1db81ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:37.665406 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:37.665356 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:37.665560 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:37.665510 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:37.665600 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:37.665576 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret podName:ff6d597e-b9e7-4074-b08e-0ab2234d4a3a nodeName:}" failed. No retries permitted until 2026-07-30 22:47:39.665560748 +0000 UTC m=+6.138024737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret") pod "global-pull-secret-syncer-spp8w" (UID: "ff6d597e-b9e7-4074-b08e-0ab2234d4a3a") : object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:37.766309 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:37.766275 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:37.766453 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:37.766431 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 22:47:37.766453 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:37.766453 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 22:47:37.766566 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:37.766463 2554 projected.go:196] Error preparing data for projected volume kube-api-access-6whhx for pod openshift-network-diagnostics/network-check-target-z72p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:37.766566 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:37.766511 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx podName:a14e59aa-d437-481b-803f-1dacc98a952c nodeName:}" failed. No retries permitted until 2026-07-30 22:47:39.76649728 +0000 UTC m=+6.238961286 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6whhx" (UniqueName: "kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx") pod "network-check-target-z72p4" (UID: "a14e59aa-d437-481b-803f-1dacc98a952c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:37.990017 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:37.989886 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 22:42:36 +0000 UTC" deadline="2028-05-01 21:08:42.009578294 +0000 UTC" Jul 30 22:47:37.990017 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:37.989958 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15382h21m4.019624792s" Jul 30 22:47:38.032752 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.032069 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:38.032752 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:38.032194 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:38.032752 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.032579 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:38.032752 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.032591 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:38.032752 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:38.032682 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:38.033128 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:38.032780 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:38.046688 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.046658 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"a3dd2a47902d71adc07070662947f2ad3cf74ec69ce0ac09473cb46db40b0eed"} Jul 30 22:47:38.049828 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.049801 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t9f9p" event={"ID":"2ab13381-4217-4fbd-a559-07f05259738f","Type":"ContainerStarted","Data":"0132d1eeb933f82d40e8c0f1ecf8338b840854d74b8577c79ca09067b60d5773"} Jul 30 22:47:38.051682 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.051647 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9srwj" event={"ID":"f340f7c6-8994-43b9-a22a-8b7a4883518c","Type":"ContainerStarted","Data":"417dd9a85d452670361ee40b28a8a95dffe8b31f4f9f08fbc233737ed5c7f1f2"} Jul 30 22:47:38.057203 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.057182 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" event={"ID":"15e562856c5ed1d4d691baee256e1db7","Type":"ContainerStarted","Data":"29ec05d2f5d9f60c469f39779f3d0ed367843321b949838a05a770f7fed4bbf5"} Jul 30 22:47:38.064349 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.064322 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" event={"ID":"ca438473-69cd-4a33-89de-580b19abe276","Type":"ContainerStarted","Data":"ecec4df4f2f674ea2ab858c5f9db2494cac7bd7203b2e90e4e81b0ba1a210549"} Jul 30 22:47:38.066658 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.066634 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerStarted","Data":"ef01e9041e8f37ff187c95b1fa87c24b21d1f84471674af33e92d06174dce61e"} Jul 30 22:47:38.084223 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.084196 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jqkvm" event={"ID":"dfd556d1-cf73-4e72-9959-988c7d67484e","Type":"ContainerStarted","Data":"4e45e5c6283035aea342aae9e0a233e44a57d4088fc6462fc4f1fcd8a678eaaa"} Jul 30 22:47:38.088475 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.088444 2554 generic.go:356] "Generic (PLEG): container finished" podID="232b7bf5bb7c90260e08f28534b57e67" containerID="a23c08ad827d5e089fdcbe486217e393307a68db3b5b020f3ba4917c8f4eb053" exitCode=0 Jul 30 22:47:38.088584 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.088526 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" event={"ID":"232b7bf5bb7c90260e08f28534b57e67","Type":"ContainerDied","Data":"a23c08ad827d5e089fdcbe486217e393307a68db3b5b020f3ba4917c8f4eb053"} Jul 30 22:47:38.091828 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.091802 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mm57w" event={"ID":"bf94eccb-32e3-45b3-9329-8991e4f7d720","Type":"ContainerStarted","Data":"a71f6a5637c54a7c01d6ba7c2b477cc660434995e2033da84cf8901a4f1f8ddc"} Jul 30 22:47:38.107526 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.107477 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-130.ec2.internal" podStartSLOduration=3.107459378 podStartE2EDuration="3.107459378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:38.073626958 +0000 UTC m=+4.546090976" watchObservedRunningTime="2026-07-30 22:47:38.107459378 +0000 UTC m=+4.579923391" Jul 30 22:47:38.107812 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.107790 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jg22k" event={"ID":"ae00cd60-e89a-4db6-af10-7004eba7c1d3","Type":"ContainerStarted","Data":"6cf6fe247fb9902b47e167f0a176d6c6d1c2b298b7cd855a70628f3ae19b43b9"} Jul 30 22:47:38.122094 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:38.122064 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hh7d2" event={"ID":"e2668b41-384e-433b-8487-35040f19364f","Type":"ContainerStarted","Data":"04a19891ba45b7d8afe16761d447e600a05384c37282323704e6b6d5b17a6e7f"} Jul 30 22:47:39.153836 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:39.153749 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" event={"ID":"232b7bf5bb7c90260e08f28534b57e67","Type":"ContainerStarted","Data":"3df5f4a6d129e127e1f750d7b0231b1911a56612b8079bf190164282753d42f3"} Jul 30 22:47:39.582305 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:39.582218 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:39.582466 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:39.582382 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:39.582466 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:39.582455 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs podName:25f75d33-3b48-4851-b5d8-5e7cd1db81ed nodeName:}" failed. No retries permitted until 2026-07-30 22:47:43.582436321 +0000 UTC m=+10.054900314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs") pod "network-metrics-daemon-2n7px" (UID: "25f75d33-3b48-4851-b5d8-5e7cd1db81ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:39.683767 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:39.683135 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:39.683767 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:39.683295 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:39.683767 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:39.683356 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret podName:ff6d597e-b9e7-4074-b08e-0ab2234d4a3a nodeName:}" failed. No retries permitted until 2026-07-30 22:47:43.68333815 +0000 UTC m=+10.155802155 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret") pod "global-pull-secret-syncer-spp8w" (UID: "ff6d597e-b9e7-4074-b08e-0ab2234d4a3a") : object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:39.784770 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:39.784294 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:39.784770 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:39.784483 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 22:47:39.784770 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:39.784503 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 22:47:39.784770 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:39.784515 2554 projected.go:196] Error preparing data for projected volume kube-api-access-6whhx for pod openshift-network-diagnostics/network-check-target-z72p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:39.784770 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:39.784571 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx podName:a14e59aa-d437-481b-803f-1dacc98a952c nodeName:}" failed. No retries permitted until 2026-07-30 22:47:43.784553035 +0000 UTC m=+10.257017029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6whhx" (UniqueName: "kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx") pod "network-check-target-z72p4" (UID: "a14e59aa-d437-481b-803f-1dacc98a952c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:40.038922 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:40.038840 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:40.039103 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:40.039012 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:40.039524 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:40.039502 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:40.039663 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:40.039643 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:40.040763 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:40.040725 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:40.040964 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:40.040942 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:42.031817 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:42.031040 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:42.031817 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:42.031171 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:42.031817 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:42.031544 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:42.031817 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:42.031648 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:42.031817 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:42.031696 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:42.031817 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:42.031784 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:43.619546 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:43.619510 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:43.620019 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:43.619636 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:43.620019 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:43.619720 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs podName:25f75d33-3b48-4851-b5d8-5e7cd1db81ed nodeName:}" failed. No retries permitted until 2026-07-30 22:47:51.619697843 +0000 UTC m=+18.092161847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs") pod "network-metrics-daemon-2n7px" (UID: "25f75d33-3b48-4851-b5d8-5e7cd1db81ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:43.721379 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:43.720826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:43.721379 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:43.720974 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:43.721379 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:43.721047 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret podName:ff6d597e-b9e7-4074-b08e-0ab2234d4a3a nodeName:}" failed. No retries permitted until 2026-07-30 22:47:51.721027841 +0000 UTC m=+18.193491848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret") pod "global-pull-secret-syncer-spp8w" (UID: "ff6d597e-b9e7-4074-b08e-0ab2234d4a3a") : object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:43.821299 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:43.821226 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:43.821460 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:43.821409 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 22:47:43.821460 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:43.821439 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 22:47:43.821460 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:43.821455 2554 projected.go:196] Error preparing data for projected volume kube-api-access-6whhx for pod openshift-network-diagnostics/network-check-target-z72p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:43.821605 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:43.821521 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx podName:a14e59aa-d437-481b-803f-1dacc98a952c nodeName:}" failed. No retries permitted until 2026-07-30 22:47:51.821501004 +0000 UTC m=+18.293964998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6whhx" (UniqueName: "kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx") pod "network-check-target-z72p4" (UID: "a14e59aa-d437-481b-803f-1dacc98a952c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:44.031433 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:44.031348 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:44.031588 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:44.031477 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:44.031588 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:44.031534 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:44.031711 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:44.031602 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:44.031824 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:44.031803 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:44.031938 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:44.031907 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:46.031021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:46.030378 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:46.031021 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:46.030506 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:46.031021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:46.030905 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:46.031021 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:46.030971 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:46.031780 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:46.031654 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:46.031780 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:46.031745 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:48.030485 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:48.030458 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:48.030485 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:48.030482 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:48.030980 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:48.030565 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:48.030980 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:48.030583 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:48.030980 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:48.030650 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:48.030980 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:48.030752 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:50.030713 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:50.030671 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:50.031150 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:50.030725 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:50.031150 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:50.030671 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:50.031150 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:50.030835 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:50.031150 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:50.030934 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:50.031150 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:50.031005 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:51.684665 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:51.684636 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:51.685119 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:51.684757 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:51.685119 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:51.684809 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs podName:25f75d33-3b48-4851-b5d8-5e7cd1db81ed nodeName:}" failed. No retries permitted until 2026-07-30 22:48:07.684793506 +0000 UTC m=+34.157257502 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs") pod "network-metrics-daemon-2n7px" (UID: "25f75d33-3b48-4851-b5d8-5e7cd1db81ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:47:51.785797 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:51.785763 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:51.785959 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:51.785917 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:51.786004 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:51.785980 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret podName:ff6d597e-b9e7-4074-b08e-0ab2234d4a3a nodeName:}" failed. No retries permitted until 2026-07-30 22:48:07.785963726 +0000 UTC m=+34.258427725 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret") pod "global-pull-secret-syncer-spp8w" (UID: "ff6d597e-b9e7-4074-b08e-0ab2234d4a3a") : object "kube-system"/"original-pull-secret" not registered Jul 30 22:47:51.887101 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:51.887068 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:51.887268 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:51.887216 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 22:47:51.887268 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:51.887234 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 22:47:51.887268 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:51.887246 2554 projected.go:196] Error preparing data for projected volume kube-api-access-6whhx for pod openshift-network-diagnostics/network-check-target-z72p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:51.887393 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:51.887305 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx podName:a14e59aa-d437-481b-803f-1dacc98a952c nodeName:}" failed. No retries permitted until 2026-07-30 22:48:07.887285633 +0000 UTC m=+34.359749633 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6whhx" (UniqueName: "kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx") pod "network-check-target-z72p4" (UID: "a14e59aa-d437-481b-803f-1dacc98a952c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:47:52.030040 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:52.029957 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:52.030040 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:52.029991 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:52.030244 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:52.029960 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:52.030244 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:52.030076 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:52.030244 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:52.030173 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:52.030391 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:52.030257 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:54.030936 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:54.030902 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:54.031393 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:54.031003 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:54.031393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:54.031113 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:54.031393 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:54.031225 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:54.031393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:54.031290 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:54.031543 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:54.031395 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:56.030768 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.030405 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:56.031334 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.030432 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:56.031334 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:56.030887 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:56.031334 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.030453 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:56.031334 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:56.031026 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:56.031334 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:56.031117 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:56.207175 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.207148 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t9f9p" event={"ID":"2ab13381-4217-4fbd-a559-07f05259738f","Type":"ContainerStarted","Data":"a5ea62828bf39dae60196b84b42457a293f745ee5d59ac417f43444d03f15235"} Jul 30 22:47:56.208791 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.208763 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9srwj" event={"ID":"f340f7c6-8994-43b9-a22a-8b7a4883518c","Type":"ContainerStarted","Data":"666e863642e5cf5a2ea0562b858900bbc310c3b702d21c618fce4a649c545b58"} Jul 30 22:47:56.210095 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.210077 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" event={"ID":"ca438473-69cd-4a33-89de-580b19abe276","Type":"ContainerStarted","Data":"9d74ae6b2486091419aacaaad2f1479fb78efe9bbe64e14d0fb9329296942306"} Jul 30 22:47:56.211503 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.211479 2554 generic.go:356] "Generic (PLEG): container finished" podID="9ff92d75-8091-4a72-a19b-6a39fef8f5d9" containerID="658ea961d5500584f1d608b8c2153b302977686e2adb46255653813aac700135" exitCode=0 Jul 30 22:47:56.211651 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.211555 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerDied","Data":"658ea961d5500584f1d608b8c2153b302977686e2adb46255653813aac700135"} Jul 30 22:47:56.213083 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.213045 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jqkvm" event={"ID":"dfd556d1-cf73-4e72-9959-988c7d67484e","Type":"ContainerStarted","Data":"2f7913f014d8a169ca92ec7a88753274b044f03aa18ec6ddda52346e92d18aeb"} Jul 30 22:47:56.214559 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.214528 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mm57w" event={"ID":"bf94eccb-32e3-45b3-9329-8991e4f7d720","Type":"ContainerStarted","Data":"5b7b53424766baad2eea39b87c827d6761959226d470d2dffb90b8d75a37f104"} Jul 30 22:47:56.215750 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.215719 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jg22k" event={"ID":"ae00cd60-e89a-4db6-af10-7004eba7c1d3","Type":"ContainerStarted","Data":"c46181e03396d9965342237e641a810bc8794b4c6dd24c2469910251fa9b1580"} Jul 30 22:47:56.219031 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.219015 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"5eb5523c124b984085ecccb474a743ab0a4512e7db4eeb8b17c2401bcc7d0e6b"} Jul 30 22:47:56.219115 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.219064 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"85a37e2ed55880280706419e24119149da65f8a812946c9de8ea7bf6bd771fe6"} Jul 30 22:47:56.219115 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.219093 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"7193c301021e20f5ab626035c754c8282b30b7acead61d9c5da024560d524f91"} Jul 30 22:47:56.219182 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.219122 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"dc001c40a5de4750a5d0ba5a34e133252435f30943b6161b2d78805ee2d074d0"} Jul 30 22:47:56.219182 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.219149 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"478b380b612b58aa1ee54376a541ad17f375da916ee52bd3f309f7e592ddb144"} Jul 30 22:47:56.222951 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.222917 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-130.ec2.internal" podStartSLOduration=21.222905451 podStartE2EDuration="21.222905451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:39.17537025 +0000 UTC m=+5.647834263" watchObservedRunningTime="2026-07-30 22:47:56.222905451 +0000 UTC m=+22.695369527" Jul 30 22:47:56.223134 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.223118 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t9f9p" podStartSLOduration=3.857418643 podStartE2EDuration="22.223114189s" totalImagesPullingTime="18.365695546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:56.222436056 +0000 UTC m=+22.694900070" watchObservedRunningTime="2026-07-30 22:47:56.223114189 +0000 UTC m=+22.695578200" Jul 30 22:47:56.262328 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.262291 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jg22k" podStartSLOduration=3.850488193 podStartE2EDuration="22.2622758s" totalImagesPullingTime="18.411787607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:56.262134313 +0000 UTC m=+22.734598326" watchObservedRunningTime="2026-07-30 22:47:56.2622758 +0000 UTC m=+22.734739812" Jul 30 22:47:56.262657 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.262638 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jqkvm" podStartSLOduration=3.799752828 podStartE2EDuration="22.262632758s" totalImagesPullingTime="18.46287993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:56.242193533 +0000 UTC m=+22.714657544" watchObservedRunningTime="2026-07-30 22:47:56.262632758 +0000 UTC m=+22.735096770" Jul 30 22:47:56.308649 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.308602 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mm57w" podStartSLOduration=3.9216568069999997 podStartE2EDuration="22.308588071s" totalImagesPullingTime="18.386931264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:56.307502972 +0000 UTC m=+22.779966985" watchObservedRunningTime="2026-07-30 22:47:56.308588071 +0000 UTC m=+22.781052084" Jul 30 22:47:56.326508 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:56.326467 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9srwj" podStartSLOduration=3.935905189 podStartE2EDuration="22.326453237s" totalImagesPullingTime="18.390548048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:56.326207425 +0000 UTC m=+22.798671437" watchObservedRunningTime="2026-07-30 22:47:56.326453237 +0000 UTC m=+22.798917249" Jul 30 22:47:57.224408 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:57.224373 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"b6f6dbd96f0e63dc79142a93d7f5b0cefb3b8343bc77cfff9160417c20d37799"} Jul 30 22:47:57.225917 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:57.225884 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hh7d2" event={"ID":"e2668b41-384e-433b-8487-35040f19364f","Type":"ContainerStarted","Data":"b428af9b3cea6819f8e548d8f2c983766958cf6fcf813c22279dded44dfb46f6"} Jul 30 22:47:57.244233 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:57.244188 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hh7d2" podStartSLOduration=4.882864867 podStartE2EDuration="23.244171493s" totalImagesPullingTime="18.361306626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:57.243364362 +0000 UTC m=+23.715828374" watchObservedRunningTime="2026-07-30 22:47:57.244171493 +0000 UTC m=+23.716635505" Jul 30 22:47:57.621331 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:57.621301 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 30 22:47:58.030102 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.030068 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:47:58.030291 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.030077 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:47:58.030291 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:58.030198 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:47:58.030291 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:58.030282 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:47:58.030486 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.030077 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:47:58.030486 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:47:58.030363 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:47:58.134099 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.133998 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T22:47:57.621327682Z","UUID":"593feaba-c408-4d4e-8277-bdba1df14956","Handler":null,"Name":"","Endpoint":""} Jul 30 22:47:58.135681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.135660 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 30 22:47:58.135834 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.135698 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 30 22:47:58.229827 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.229792 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" event={"ID":"ca438473-69cd-4a33-89de-580b19abe276","Type":"ContainerStarted","Data":"f566d58b9850887e1196d4c1b64db8b189d9fcb3afe203f53064fc55055c4b81"} Jul 30 22:47:58.524209 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.524182 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:58.524653 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:58.524634 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:59.236042 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:59.236008 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"b84a81a51d97a7115eae9f0d527ead18594a5a95e8486acb2795a0abaa411d01"} Jul 30 22:47:59.236500 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:59.236109 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:47:59.236681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:47:59.236667 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t9f9p" Jul 30 22:48:00.030943 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:00.030745 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:00.031133 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:00.030750 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:00.031133 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:00.031058 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:48:00.031133 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:00.030809 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:00.031133 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:00.031119 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:48:00.031340 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:00.031222 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:48:01.241613 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.241358 2554 generic.go:356] "Generic (PLEG): container finished" podID="9ff92d75-8091-4a72-a19b-6a39fef8f5d9" containerID="c522f6ed574c2a0f275865972d3779ed9ce6298667978befb559e743b223743c" exitCode=0 Jul 30 22:48:01.242453 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.241437 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerDied","Data":"c522f6ed574c2a0f275865972d3779ed9ce6298667978befb559e743b223743c"} Jul 30 22:48:01.245276 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.245251 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" event={"ID":"ce1d4419-eb7f-4e08-8a7b-824a9baf39ee","Type":"ContainerStarted","Data":"3390b5840237fd77a6405f63aa156d5dee79e1662a8c008939bf7d1b36aefe1c"} Jul 30 22:48:01.245496 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.245476 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:48:01.245625 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.245539 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:48:01.247708 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.247685 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" event={"ID":"ca438473-69cd-4a33-89de-580b19abe276","Type":"ContainerStarted","Data":"7ea6af535120fb53500d2304a8a373b92df4430a10db0072f8bb4c765884b0f8"} Jul 30 22:48:01.261998 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.261972 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:48:01.262118 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.262105 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:48:01.312467 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.312413 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" podStartSLOduration=8.856476125 podStartE2EDuration="27.312396817s" totalImagesPullingTime="18.455920692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:01.311298574 +0000 UTC m=+27.783762585" watchObservedRunningTime="2026-07-30 22:48:01.312396817 +0000 UTC m=+27.784860829" Jul 30 22:48:01.312636 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:01.312557 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kf7nz" podStartSLOduration=4.322603872 podStartE2EDuration="27.312553158s" totalImagesPullingTime="22.989949286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:01.284188374 +0000 UTC m=+27.756652386" watchObservedRunningTime="2026-07-30 22:48:01.312553158 +0000 UTC m=+27.785017170" Jul 30 22:48:02.030978 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:02.030899 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:02.031146 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:02.030899 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:02.031146 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:02.031035 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:48:02.031146 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:02.030909 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:02.031146 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:02.031121 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:48:02.031344 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:02.031214 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:48:02.249964 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:02.249934 2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 30 22:48:03.254075 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.254043 2554 generic.go:356] "Generic (PLEG): container finished" podID="9ff92d75-8091-4a72-a19b-6a39fef8f5d9" containerID="932783dcbaffbdb94ca705f444d49b10dce7f5a5250cd6a217ded03ddd715d22" exitCode=0 Jul 30 22:48:03.254569 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.254138 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerDied","Data":"932783dcbaffbdb94ca705f444d49b10dce7f5a5250cd6a217ded03ddd715d22"} Jul 30 22:48:03.254569 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.254170 2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 30 22:48:03.284281 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.284246 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-spp8w"] Jul 30 22:48:03.284447 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.284347 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:03.284447 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:03.284429 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:48:03.290813 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.290784 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z72p4"] Jul 30 22:48:03.290959 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.290872 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:03.290959 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:03.290944 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:48:03.292843 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.292818 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2n7px"] Jul 30 22:48:03.292961 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.292905 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:03.293002 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:03.292990 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:48:03.540851 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:03.540765 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:48:05.030650 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:05.030618 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:05.031059 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:05.030717 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:05.031059 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:05.030725 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:48:05.031059 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:05.030759 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:05.031059 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:05.030828 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:48:05.031059 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:05.030892 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:48:05.261393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:05.261201 2554 generic.go:356] "Generic (PLEG): container finished" podID="9ff92d75-8091-4a72-a19b-6a39fef8f5d9" containerID="80abb39675d09a786d1ec7eafcd604c1705eaf6823d55187641800595b5004b6" exitCode=0 Jul 30 22:48:05.261566 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:05.261280 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerDied","Data":"80abb39675d09a786d1ec7eafcd604c1705eaf6823d55187641800595b5004b6"} Jul 30 22:48:07.029794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:07.029708 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:07.030170 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:07.029818 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:07.030170 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.029825 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-spp8w" podUID="ff6d597e-b9e7-4074-b08e-0ab2234d4a3a" Jul 30 22:48:07.030170 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:07.029847 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:07.030170 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.029923 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z72p4" podUID="a14e59aa-d437-481b-803f-1dacc98a952c" Jul 30 22:48:07.030170 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.029992 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2n7px" podUID="25f75d33-3b48-4851-b5d8-5e7cd1db81ed" Jul 30 22:48:07.708220 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:07.708181 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:07.708412 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.708304 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:48:07.708412 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.708360 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs podName:25f75d33-3b48-4851-b5d8-5e7cd1db81ed nodeName:}" failed. No retries permitted until 2026-07-30 22:48:39.708345455 +0000 UTC m=+66.180809446 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs") pod "network-metrics-daemon-2n7px" (UID: "25f75d33-3b48-4851-b5d8-5e7cd1db81ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 22:48:07.808956 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:07.808923 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:07.809145 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.809109 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 22:48:07.809215 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.809186 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret podName:ff6d597e-b9e7-4074-b08e-0ab2234d4a3a nodeName:}" failed. No retries permitted until 2026-07-30 22:48:39.809165099 +0000 UTC m=+66.281629103 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret") pod "global-pull-secret-syncer-spp8w" (UID: "ff6d597e-b9e7-4074-b08e-0ab2234d4a3a") : object "kube-system"/"original-pull-secret" not registered Jul 30 22:48:07.909916 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:07.909874 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:07.910084 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.910004 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 22:48:07.910084 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.910027 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 22:48:07.910084 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.910039 2554 projected.go:196] Error preparing data for projected volume kube-api-access-6whhx for pod openshift-network-diagnostics/network-check-target-z72p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:48:07.910197 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:07.910101 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx podName:a14e59aa-d437-481b-803f-1dacc98a952c nodeName:}" failed. No retries permitted until 2026-07-30 22:48:39.910086464 +0000 UTC m=+66.382550456 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6whhx" (UniqueName: "kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx") pod "network-check-target-z72p4" (UID: "a14e59aa-d437-481b-803f-1dacc98a952c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 22:48:08.346652 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.346620 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-130.ec2.internal" event="NodeReady" Jul 30 22:48:08.347082 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.346791 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 30 22:48:08.386675 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.386643 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474"] Jul 30 22:48:08.390227 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.390203 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64b7cb797d-rmcrv"] Jul 30 22:48:08.390466 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.390442 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.393336 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.393316 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.396532 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.396513 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 30 22:48:08.397001 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.396976 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.397203 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.397186 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.397346 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.397328 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 30 22:48:08.397625 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.397604 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2qk4h\"" Jul 30 22:48:08.397718 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.397634 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 30 22:48:08.397718 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.397645 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 30 22:48:08.397718 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.397644 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m4zn7\"" Jul 30 22:48:08.399954 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.399936 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 30 22:48:08.405497 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.405470 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp"] Jul 30 22:48:08.408617 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.408465 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 30 22:48:08.410342 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.410322 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474"] Jul 30 22:48:08.410428 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.410360 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv"] Jul 30 22:48:08.410493 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.410435 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.413308 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.413129 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.414181 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.414161 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sz7mb\"" Jul 30 22:48:08.414329 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.414271 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.414329 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.414302 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 30 22:48:08.414521 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.414414 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 30 22:48:08.414697 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.414681 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-bfq27"] Jul 30 22:48:08.415578 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.415560 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.418555 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.418517 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6"] Jul 30 22:48:08.419025 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.419004 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.421665 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.421644 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.421788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.421712 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.421879 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.421860 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 30 22:48:08.421941 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.421868 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7gl4r\"" Jul 30 22:48:08.422184 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.422168 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 30 22:48:08.423814 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.423795 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.423954 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.423937 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 30 22:48:08.424168 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.424148 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.424263 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.424214 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2ks9f\"" Jul 30 22:48:08.426913 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.426892 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw"] Jul 30 22:48:08.427003 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.426990 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:08.428312 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.428294 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 30 22:48:08.428663 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.428647 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 30 22:48:08.429813 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.429793 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.429813 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.429808 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.429952 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.429880 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 30 22:48:08.430009 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.429957 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pj2kd\"" Jul 30 22:48:08.430161 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.430144 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-bzz2p"] Jul 30 22:48:08.430277 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.430263 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw" Jul 30 22:48:08.432471 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.432454 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.432471 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.432477 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nqznz\"" Jul 30 22:48:08.432645 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.432627 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.433084 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.433067 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq"] Jul 30 22:48:08.433191 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.433174 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.436387 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.436370 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.436511 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.436408 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.436979 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-66f84c8f94-gds4x"] Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.437507 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.438300 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.438640 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq" Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.440350 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64b7cb797d-rmcrv"] Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.440383 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp"] Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.440560 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.441549 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-bfq27"] Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.443515 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv"] Jul 30 22:48:08.444275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.444000 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-p9vq7\"" Jul 30 22:48:08.444890 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.444340 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fg66p"] Jul 30 22:48:08.445477 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.445458 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 30 22:48:08.445642 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.445623 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-65m5w\"" Jul 30 22:48:08.445783 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.445489 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.445862 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.445555 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.445939 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.445887 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 30 22:48:08.446251 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.446235 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 30 22:48:08.446519 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.446504 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c9nx6\"" Jul 30 22:48:08.446970 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.446948 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.447270 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.447243 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.447645 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.447618 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 30 22:48:08.450648 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.450629 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 30 22:48:08.451706 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.451690 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d8qbt"] Jul 30 22:48:08.452155 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.452135 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.455140 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.455122 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6"] Jul 30 22:48:08.455300 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.455286 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw"] Jul 30 22:48:08.455423 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.455411 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-66f84c8f94-gds4x"] Jul 30 22:48:08.455523 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.455502 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 30 22:48:08.455610 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.455525 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 30 22:48:08.455610 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.455222 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d8qbt" Jul 30 22:48:08.455610 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.455600 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g5cvn\"" Jul 30 22:48:08.461506 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.461485 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zlg2h\"" Jul 30 22:48:08.461928 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.461911 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 30 22:48:08.462338 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.462322 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq"] Jul 30 22:48:08.463098 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.463047 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 30 22:48:08.463384 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.463365 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 30 22:48:08.463576 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.463561 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fg66p"] Jul 30 22:48:08.464934 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.464910 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-bzz2p"] Jul 30 22:48:08.479334 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.479308 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d8qbt"] Jul 30 22:48:08.513348 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513329 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsckz\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-kube-api-access-zsckz\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.513537 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513513 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6ksz\" (UniqueName: \"kubernetes.io/projected/05000c78-03e9-4364-86c2-5aa37dd2dd14-kube-api-access-m6ksz\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.513636 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513548 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.513636 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513572 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbwm8\" (UniqueName: \"kubernetes.io/projected/161408ab-5872-4ef0-a41c-93bf201f1e3c-kube-api-access-qbwm8\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.513636 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513600 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-ca-trust-extracted\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.513636 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513625 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/161408ab-5872-4ef0-a41c-93bf201f1e3c-config\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.513833 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513655 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-certificates\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.513833 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513681 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-bound-sa-token\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.513833 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513759 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05000c78-03e9-4364-86c2-5aa37dd2dd14-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.513958 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513827 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-installation-pull-secrets\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.513958 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513881 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90fd9a18-a33a-45c5-b685-20e709a56cf7-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.513958 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513901 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ba57d39-8636-40b7-b529-906527b1c207-config\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.513958 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513923 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05000c78-03e9-4364-86c2-5aa37dd2dd14-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.513958 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513946 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng6s6\" (UniqueName: \"kubernetes.io/projected/90fd9a18-a33a-45c5-b685-20e709a56cf7-kube-api-access-ng6s6\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.514177 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.513972 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-image-registry-private-configuration\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.514177 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.514007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ba57d39-8636-40b7-b529-906527b1c207-serving-cert\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.514177 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.514028 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ba57d39-8636-40b7-b529-906527b1c207-trusted-ca\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.514177 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.514058 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.514177 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.514081 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxwtc\" (UniqueName: \"kubernetes.io/projected/5ba57d39-8636-40b7-b529-906527b1c207-kube-api-access-sxwtc\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.514391 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.514187 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-trusted-ca\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.514391 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.514224 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/161408ab-5872-4ef0-a41c-93bf201f1e3c-serving-cert\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.615108 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615073 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05000c78-03e9-4364-86c2-5aa37dd2dd14-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.615108 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615112 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-bound-sa-token\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.615337 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615141 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fce2a95b-b468-4a4d-bf2f-76e040945724-tmp\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.615337 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615162 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-installation-pull-secrets\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.615337 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615181 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgmxf\" (UniqueName: \"kubernetes.io/projected/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-kube-api-access-hgmxf\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.615337 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615204 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghkkz\" (UniqueName: \"kubernetes.io/projected/244bddc1-aaf0-4e98-98c5-5302a60800a6-kube-api-access-ghkkz\") pod \"network-check-source-65bdbbff45-g4fxw\" (UID: \"244bddc1-aaf0-4e98-98c5-5302a60800a6\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw" Jul 30 22:48:08.615337 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615268 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90fd9a18-a33a-45c5-b685-20e709a56cf7-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.615337 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615291 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxwtc\" (UniqueName: \"kubernetes.io/projected/5ba57d39-8636-40b7-b529-906527b1c207-kube-api-access-sxwtc\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.615337 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615326 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fce2a95b-b468-4a4d-bf2f-76e040945724-serving-cert\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.615635 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615353 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ba57d39-8636-40b7-b529-906527b1c207-serving-cert\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.615635 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615383 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-tmp-dir\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.615635 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615607 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5nzv\" (UniqueName: \"kubernetes.io/projected/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-kube-api-access-c5nzv\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.615781 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615664 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-default-certificate\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.615781 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615717 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.615781 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615765 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.615929 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615796 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhd29\" (UniqueName: \"kubernetes.io/projected/1d8434f7-e903-4268-aae4-fe78c1ab339e-kube-api-access-fhd29\") pod \"ingress-canary-d8qbt\" (UID: \"1d8434f7-e903-4268-aae4-fe78c1ab339e\") " pod="openshift-ingress-canary/ingress-canary-d8qbt" Jul 30 22:48:08.615929 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615853 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/161408ab-5872-4ef0-a41c-93bf201f1e3c-config\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.615929 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.615880 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 22:48:08.615929 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.615895 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b7cb797d-rmcrv: secret "image-registry-tls" not found Jul 30 22:48:08.616084 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.615972 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls podName:b4b22577-b8ad-4dd9-88a1-365e5bf69c27 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:09.115954764 +0000 UTC m=+35.588418755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls") pod "image-registry-64b7cb797d-rmcrv" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27") : secret "image-registry-tls" not found Jul 30 22:48:08.616084 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616039 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90fd9a18-a33a-45c5-b685-20e709a56cf7-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.616084 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.615881 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:08.616178 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616092 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.616178 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616122 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-certificates\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.616178 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616150 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbzd9\" (UniqueName: \"kubernetes.io/projected/fce2a95b-b468-4a4d-bf2f-76e040945724-kube-api-access-cbzd9\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.616300 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616175 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmdw4\" (UniqueName: \"kubernetes.io/projected/93654742-1acd-4b8c-97ca-1cf1d5c73570-kube-api-access-xmdw4\") pod \"volume-data-source-validator-5b8776669-bx5lq\" (UID: \"93654742-1acd-4b8c-97ca-1cf1d5c73570\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq" Jul 30 22:48:08.616300 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616211 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ba57d39-8636-40b7-b529-906527b1c207-config\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.616300 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616238 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7x7r\" (UniqueName: \"kubernetes.io/projected/515ff8e0-d3c7-41b5-8f17-5acd0df58061-kube-api-access-l7x7r\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:08.616300 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616262 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05000c78-03e9-4364-86c2-5aa37dd2dd14-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.616300 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616287 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng6s6\" (UniqueName: \"kubernetes.io/projected/90fd9a18-a33a-45c5-b685-20e709a56cf7-kube-api-access-ng6s6\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.616465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616317 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-image-registry-private-configuration\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.616465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616356 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d8434f7-e903-4268-aae4-fe78c1ab339e-cert\") pod \"ingress-canary-d8qbt\" (UID: \"1d8434f7-e903-4268-aae4-fe78c1ab339e\") " pod="openshift-ingress-canary/ingress-canary-d8qbt" Jul 30 22:48:08.616465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616362 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/161408ab-5872-4ef0-a41c-93bf201f1e3c-config\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.616465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616384 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ba57d39-8636-40b7-b529-906527b1c207-trusted-ca\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.616465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616410 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce2a95b-b468-4a4d-bf2f-76e040945724-trusted-ca-bundle\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.616465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616435 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fce2a95b-b468-4a4d-bf2f-76e040945724-snapshots\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.616465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616459 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-config-volume\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616491 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-trusted-ca\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616518 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/161408ab-5872-4ef0-a41c-93bf201f1e3c-serving-cert\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616543 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsckz\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-kube-api-access-zsckz\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616571 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce2a95b-b468-4a4d-bf2f-76e040945724-service-ca-bundle\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616598 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616649 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ksz\" (UniqueName: \"kubernetes.io/projected/05000c78-03e9-4364-86c2-5aa37dd2dd14-kube-api-access-m6ksz\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616675 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbwm8\" (UniqueName: \"kubernetes.io/projected/161408ab-5872-4ef0-a41c-93bf201f1e3c-kube-api-access-qbwm8\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616705 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616752 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-stats-auth\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.616794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616781 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-ca-trust-extracted\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.617062 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.617004 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ba57d39-8636-40b7-b529-906527b1c207-config\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.617110 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.617087 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-ca-trust-extracted\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.617288 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.617265 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05000c78-03e9-4364-86c2-5aa37dd2dd14-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.617402 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.617385 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:08.617462 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.617447 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls podName:90fd9a18-a33a-45c5-b685-20e709a56cf7 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:09.117431627 +0000 UTC m=+35.589895634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-vh474" (UID: "90fd9a18-a33a-45c5-b685-20e709a56cf7") : secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:08.617462 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.616705 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-certificates\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.617778 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.617755 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ba57d39-8636-40b7-b529-906527b1c207-trusted-ca\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.618485 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.618447 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-trusted-ca\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.620112 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.620083 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:08.620422 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.620404 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ba57d39-8636-40b7-b529-906527b1c207-serving-cert\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.620514 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.620480 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-image-registry-private-configuration\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.620858 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.620836 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05000c78-03e9-4364-86c2-5aa37dd2dd14-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.620940 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.620893 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/161408ab-5872-4ef0-a41c-93bf201f1e3c-serving-cert\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.621235 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.621216 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-installation-pull-secrets\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.627487 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.627465 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-bound-sa-token\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.629250 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.629225 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxwtc\" (UniqueName: \"kubernetes.io/projected/5ba57d39-8636-40b7-b529-906527b1c207-kube-api-access-sxwtc\") pod \"console-operator-5ccd6b445f-bfq27\" (UID: \"5ba57d39-8636-40b7-b529-906527b1c207\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.633236 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.633212 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ksz\" (UniqueName: \"kubernetes.io/projected/05000c78-03e9-4364-86c2-5aa37dd2dd14-kube-api-access-m6ksz\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bcnkp\" (UID: \"05000c78-03e9-4364-86c2-5aa37dd2dd14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.633404 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.633348 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng6s6\" (UniqueName: \"kubernetes.io/projected/90fd9a18-a33a-45c5-b685-20e709a56cf7-kube-api-access-ng6s6\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:08.633526 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.633407 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbwm8\" (UniqueName: \"kubernetes.io/projected/161408ab-5872-4ef0-a41c-93bf201f1e3c-kube-api-access-qbwm8\") pod \"service-ca-operator-784f97bcf6-r5xsv\" (UID: \"161408ab-5872-4ef0-a41c-93bf201f1e3c\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.634799 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.634758 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsckz\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-kube-api-access-zsckz\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:08.717944 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.717904 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d8434f7-e903-4268-aae4-fe78c1ab339e-cert\") pod \"ingress-canary-d8qbt\" (UID: \"1d8434f7-e903-4268-aae4-fe78c1ab339e\") " pod="openshift-ingress-canary/ingress-canary-d8qbt" Jul 30 22:48:08.717944 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.717956 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce2a95b-b468-4a4d-bf2f-76e040945724-trusted-ca-bundle\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.718183 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.717984 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fce2a95b-b468-4a4d-bf2f-76e040945724-snapshots\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.718183 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718113 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-config-volume\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.718183 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718164 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce2a95b-b468-4a4d-bf2f-76e040945724-service-ca-bundle\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.718343 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718191 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.718343 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718252 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-stats-auth\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.718343 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718285 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fce2a95b-b468-4a4d-bf2f-76e040945724-tmp\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.718343 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718310 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgmxf\" (UniqueName: \"kubernetes.io/projected/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-kube-api-access-hgmxf\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.718343 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718331 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghkkz\" (UniqueName: \"kubernetes.io/projected/244bddc1-aaf0-4e98-98c5-5302a60800a6-kube-api-access-ghkkz\") pod \"network-check-source-65bdbbff45-g4fxw\" (UID: \"244bddc1-aaf0-4e98-98c5-5302a60800a6\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw" Jul 30 22:48:08.718572 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718372 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fce2a95b-b468-4a4d-bf2f-76e040945724-serving-cert\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.718572 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718402 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-tmp-dir\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.718572 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718427 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5nzv\" (UniqueName: \"kubernetes.io/projected/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-kube-api-access-c5nzv\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.718721 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718629 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-default-certificate\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.718721 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718644 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fce2a95b-b468-4a4d-bf2f-76e040945724-snapshots\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.718721 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718679 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.718721 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhd29\" (UniqueName: \"kubernetes.io/projected/1d8434f7-e903-4268-aae4-fe78c1ab339e-kube-api-access-fhd29\") pod \"ingress-canary-d8qbt\" (UID: \"1d8434f7-e903-4268-aae4-fe78c1ab339e\") " pod="openshift-ingress-canary/ingress-canary-d8qbt" Jul 30 22:48:08.718881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718767 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:08.718881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718786 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-config-volume\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.718881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.718798 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.719002 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.718885 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 22:48:08.719002 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.718946 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:09.218930885 +0000 UTC m=+35.691394877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : secret "router-metrics-certs-default" not found Jul 30 22:48:08.719100 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.719007 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce2a95b-b468-4a4d-bf2f-76e040945724-trusted-ca-bundle\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.719100 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.719061 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:09.219044894 +0000 UTC m=+35.691508916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : configmap references non-existent config key: service-ca.crt Jul 30 22:48:08.719207 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.719186 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 22:48:08.719258 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.719230 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls podName:515ff8e0-d3c7-41b5-8f17-5acd0df58061 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:09.219216506 +0000 UTC m=+35.691680498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9csh6" (UID: "515ff8e0-d3c7-41b5-8f17-5acd0df58061") : secret "samples-operator-tls" not found Jul 30 22:48:08.719258 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.719242 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce2a95b-b468-4a4d-bf2f-76e040945724-service-ca-bundle\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.719356 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.719246 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-tmp-dir\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.719356 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.719298 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbzd9\" (UniqueName: \"kubernetes.io/projected/fce2a95b-b468-4a4d-bf2f-76e040945724-kube-api-access-cbzd9\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.719356 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.719321 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 22:48:08.719356 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.719328 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmdw4\" (UniqueName: \"kubernetes.io/projected/93654742-1acd-4b8c-97ca-1cf1d5c73570-kube-api-access-xmdw4\") pod \"volume-data-source-validator-5b8776669-bx5lq\" (UID: \"93654742-1acd-4b8c-97ca-1cf1d5c73570\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq" Jul 30 22:48:08.719541 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:08.719365 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls podName:4bef5592-9ae3-4e19-9d0e-b309bceb1f59 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:09.219351646 +0000 UTC m=+35.691815651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls") pod "dns-default-fg66p" (UID: "4bef5592-9ae3-4e19-9d0e-b309bceb1f59") : secret "dns-default-metrics-tls" not found Jul 30 22:48:08.719541 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.719395 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7x7r\" (UniqueName: \"kubernetes.io/projected/515ff8e0-d3c7-41b5-8f17-5acd0df58061-kube-api-access-l7x7r\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:08.719541 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.719469 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fce2a95b-b468-4a4d-bf2f-76e040945724-tmp\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.721677 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.721609 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fce2a95b-b468-4a4d-bf2f-76e040945724-serving-cert\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.721822 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.721674 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-stats-auth\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.721822 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.721674 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-default-certificate\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.721822 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.721816 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d8434f7-e903-4268-aae4-fe78c1ab339e-cert\") pod \"ingress-canary-d8qbt\" (UID: \"1d8434f7-e903-4268-aae4-fe78c1ab339e\") " pod="openshift-ingress-canary/ingress-canary-d8qbt" Jul 30 22:48:08.729785 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.729708 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" Jul 30 22:48:08.738306 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.738285 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" Jul 30 22:48:08.741243 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.741104 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5nzv\" (UniqueName: \"kubernetes.io/projected/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-kube-api-access-c5nzv\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:08.741986 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.741967 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghkkz\" (UniqueName: \"kubernetes.io/projected/244bddc1-aaf0-4e98-98c5-5302a60800a6-kube-api-access-ghkkz\") pod \"network-check-source-65bdbbff45-g4fxw\" (UID: \"244bddc1-aaf0-4e98-98c5-5302a60800a6\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw" Jul 30 22:48:08.742330 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.742304 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7x7r\" (UniqueName: \"kubernetes.io/projected/515ff8e0-d3c7-41b5-8f17-5acd0df58061-kube-api-access-l7x7r\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:08.743263 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.743243 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhd29\" (UniqueName: \"kubernetes.io/projected/1d8434f7-e903-4268-aae4-fe78c1ab339e-kube-api-access-fhd29\") pod \"ingress-canary-d8qbt\" (UID: \"1d8434f7-e903-4268-aae4-fe78c1ab339e\") " pod="openshift-ingress-canary/ingress-canary-d8qbt" Jul 30 22:48:08.747651 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.747630 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgmxf\" (UniqueName: \"kubernetes.io/projected/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-kube-api-access-hgmxf\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:08.754455 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.754426 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:08.757871 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.757844 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:08.776902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.776871 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw" Jul 30 22:48:08.788112 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.788086 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbzd9\" (UniqueName: \"kubernetes.io/projected/fce2a95b-b468-4a4d-bf2f-76e040945724-kube-api-access-cbzd9\") pod \"insights-operator-77696685b-bzz2p\" (UID: \"fce2a95b-b468-4a4d-bf2f-76e040945724\") " pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:08.789551 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.789533 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmdw4\" (UniqueName: \"kubernetes.io/projected/93654742-1acd-4b8c-97ca-1cf1d5c73570-kube-api-access-xmdw4\") pod \"volume-data-source-validator-5b8776669-bx5lq\" (UID: \"93654742-1acd-4b8c-97ca-1cf1d5c73570\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq" Jul 30 22:48:08.791245 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.791226 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq" Jul 30 22:48:08.816497 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:08.816470 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d8qbt" Jul 30 22:48:09.029908 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.029822 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:09.029908 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.029838 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:09.030100 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.029943 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:09.032869 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.032838 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 22:48:09.034761 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.033613 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ghwvz\"" Jul 30 22:48:09.034761 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.033922 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 22:48:09.034761 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.034169 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6fgjp\"" Jul 30 22:48:09.084124 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.084081 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-bzz2p" Jul 30 22:48:09.126376 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.126344 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:09.126551 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.126437 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:09.126551 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.126524 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:09.126635 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.126574 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 22:48:09.126635 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.126589 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b7cb797d-rmcrv: secret "image-registry-tls" not found Jul 30 22:48:09.126635 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.126607 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls podName:90fd9a18-a33a-45c5-b685-20e709a56cf7 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:10.126587006 +0000 UTC m=+36.599050999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-vh474" (UID: "90fd9a18-a33a-45c5-b685-20e709a56cf7") : secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:09.126791 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.126636 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls podName:b4b22577-b8ad-4dd9-88a1-365e5bf69c27 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:10.126620727 +0000 UTC m=+36.599084720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls") pod "image-registry-64b7cb797d-rmcrv" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27") : secret "image-registry-tls" not found Jul 30 22:48:09.227184 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.227144 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:09.227348 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.227213 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:09.227348 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.227238 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:09.227348 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:09.227294 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:09.227348 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.227330 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 22:48:09.227569 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.227404 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls podName:4bef5592-9ae3-4e19-9d0e-b309bceb1f59 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:10.22738361 +0000 UTC m=+36.699847605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls") pod "dns-default-fg66p" (UID: "4bef5592-9ae3-4e19-9d0e-b309bceb1f59") : secret "dns-default-metrics-tls" not found Jul 30 22:48:09.227569 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.227408 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 22:48:09.227569 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.227423 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:10.227413931 +0000 UTC m=+36.699877928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : configmap references non-existent config key: service-ca.crt Jul 30 22:48:09.227569 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.227478 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:10.227460921 +0000 UTC m=+36.699924928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : secret "router-metrics-certs-default" not found Jul 30 22:48:09.227569 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.227491 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 22:48:09.227569 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:09.227558 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls podName:515ff8e0-d3c7-41b5-8f17-5acd0df58061 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:10.227542283 +0000 UTC m=+36.700006293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9csh6" (UID: "515ff8e0-d3c7-41b5-8f17-5acd0df58061") : secret "samples-operator-tls" not found Jul 30 22:48:10.144770 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.144614 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:10.144770 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.144721 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:10.146746 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.145958 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:10.146746 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.146273 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls podName:90fd9a18-a33a-45c5-b685-20e709a56cf7 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:12.146253399 +0000 UTC m=+38.618717409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-vh474" (UID: "90fd9a18-a33a-45c5-b685-20e709a56cf7") : secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:10.146746 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.146038 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 22:48:10.146746 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.146654 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b7cb797d-rmcrv: secret "image-registry-tls" not found Jul 30 22:48:10.146746 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.146704 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls podName:b4b22577-b8ad-4dd9-88a1-365e5bf69c27 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:12.146686306 +0000 UTC m=+38.619150297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls") pod "image-registry-64b7cb797d-rmcrv" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27") : secret "image-registry-tls" not found Jul 30 22:48:10.193702 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.193669 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq"] Jul 30 22:48:10.196807 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:10.196782 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93654742_1acd_4b8c_97ca_1cf1d5c73570.slice/crio-96cbb3056a5743b5b7f668c4f02d82283f8efadbad63b63b1a2f62f08dfb6391 WatchSource:0}: Error finding container 96cbb3056a5743b5b7f668c4f02d82283f8efadbad63b63b1a2f62f08dfb6391: Status 404 returned error can't find the container with id 96cbb3056a5743b5b7f668c4f02d82283f8efadbad63b63b1a2f62f08dfb6391 Jul 30 22:48:10.205430 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.205400 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-bfq27"] Jul 30 22:48:10.207940 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.206404 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp"] Jul 30 22:48:10.208594 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.208511 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw"] Jul 30 22:48:10.209913 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.209781 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv"] Jul 30 22:48:10.212041 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.210925 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-bzz2p"] Jul 30 22:48:10.212095 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.212058 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d8qbt"] Jul 30 22:48:10.213899 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:10.213684 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244bddc1_aaf0_4e98_98c5_5302a60800a6.slice/crio-893412602bcc1a00f203bb4bb4a92b439624dfdec58a8a3d46866403e3c633a1 WatchSource:0}: Error finding container 893412602bcc1a00f203bb4bb4a92b439624dfdec58a8a3d46866403e3c633a1: Status 404 returned error can't find the container with id 893412602bcc1a00f203bb4bb4a92b439624dfdec58a8a3d46866403e3c633a1 Jul 30 22:48:10.214018 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:10.213981 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba57d39_8636_40b7_b529_906527b1c207.slice/crio-921789cf64a697590ed2348a376c16ff68884305de867c9dd75f2a82b355628e WatchSource:0}: Error finding container 921789cf64a697590ed2348a376c16ff68884305de867c9dd75f2a82b355628e: Status 404 returned error can't find the container with id 921789cf64a697590ed2348a376c16ff68884305de867c9dd75f2a82b355628e Jul 30 22:48:10.215211 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:10.215063 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05000c78_03e9_4364_86c2_5aa37dd2dd14.slice/crio-493b33232fb9af356f224a33b0c950ff04d59cd36d7ea6c848f55db7a028190b WatchSource:0}: Error finding container 493b33232fb9af356f224a33b0c950ff04d59cd36d7ea6c848f55db7a028190b: Status 404 returned error can't find the container with id 493b33232fb9af356f224a33b0c950ff04d59cd36d7ea6c848f55db7a028190b Jul 30 22:48:10.218201 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:10.217020 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce2a95b_b468_4a4d_bf2f_76e040945724.slice/crio-b232e42eb00ea3f1c09bd8909578a725dbf1f776eeba4306325744d716993639 WatchSource:0}: Error finding container b232e42eb00ea3f1c09bd8909578a725dbf1f776eeba4306325744d716993639: Status 404 returned error can't find the container with id b232e42eb00ea3f1c09bd8909578a725dbf1f776eeba4306325744d716993639 Jul 30 22:48:10.218201 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:10.218000 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161408ab_5872_4ef0_a41c_93bf201f1e3c.slice/crio-e440fcd866e6eabb8085240ac771d3f9c6781ae1c85c9aca0054a747b19ed065 WatchSource:0}: Error finding container e440fcd866e6eabb8085240ac771d3f9c6781ae1c85c9aca0054a747b19ed065: Status 404 returned error can't find the container with id e440fcd866e6eabb8085240ac771d3f9c6781ae1c85c9aca0054a747b19ed065 Jul 30 22:48:10.246868 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.246832 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:10.247040 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.246887 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:10.247040 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.246910 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:10.247040 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.246956 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:10.247040 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.246970 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 22:48:10.247040 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.247041 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls podName:4bef5592-9ae3-4e19-9d0e-b309bceb1f59 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:12.247020924 +0000 UTC m=+38.719484927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls") pod "dns-default-fg66p" (UID: "4bef5592-9ae3-4e19-9d0e-b309bceb1f59") : secret "dns-default-metrics-tls" not found Jul 30 22:48:10.247237 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.247046 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 22:48:10.247237 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.247052 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 22:48:10.247237 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.247067 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:12.247056367 +0000 UTC m=+38.719520358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : configmap references non-existent config key: service-ca.crt Jul 30 22:48:10.247237 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.247082 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:12.247073886 +0000 UTC m=+38.719537880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : secret "router-metrics-certs-default" not found Jul 30 22:48:10.247237 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:10.247106 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls podName:515ff8e0-d3c7-41b5-8f17-5acd0df58061 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:12.247090044 +0000 UTC m=+38.719554049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9csh6" (UID: "515ff8e0-d3c7-41b5-8f17-5acd0df58061") : secret "samples-operator-tls" not found Jul 30 22:48:10.274790 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.274759 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" event={"ID":"5ba57d39-8636-40b7-b529-906527b1c207","Type":"ContainerStarted","Data":"921789cf64a697590ed2348a376c16ff68884305de867c9dd75f2a82b355628e"} Jul 30 22:48:10.275980 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.275955 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq" event={"ID":"93654742-1acd-4b8c-97ca-1cf1d5c73570","Type":"ContainerStarted","Data":"96cbb3056a5743b5b7f668c4f02d82283f8efadbad63b63b1a2f62f08dfb6391"} Jul 30 22:48:10.278916 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.278878 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerStarted","Data":"8193f8a938c9cff375c2888e4ea6f7a0fdbacea8bad6a890713c969fcb3500bf"} Jul 30 22:48:10.280355 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.280335 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" event={"ID":"05000c78-03e9-4364-86c2-5aa37dd2dd14","Type":"ContainerStarted","Data":"493b33232fb9af356f224a33b0c950ff04d59cd36d7ea6c848f55db7a028190b"} Jul 30 22:48:10.281548 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.281520 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-bzz2p" event={"ID":"fce2a95b-b468-4a4d-bf2f-76e040945724","Type":"ContainerStarted","Data":"b232e42eb00ea3f1c09bd8909578a725dbf1f776eeba4306325744d716993639"} Jul 30 22:48:10.282822 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.282757 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d8qbt" event={"ID":"1d8434f7-e903-4268-aae4-fe78c1ab339e","Type":"ContainerStarted","Data":"d1377ac9f8aca26c80866707f5795833efcd73fe85a8599be65f90c24a1bf5ab"} Jul 30 22:48:10.284039 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.284017 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" event={"ID":"161408ab-5872-4ef0-a41c-93bf201f1e3c","Type":"ContainerStarted","Data":"e440fcd866e6eabb8085240ac771d3f9c6781ae1c85c9aca0054a747b19ed065"} Jul 30 22:48:10.285098 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:10.285077 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw" event={"ID":"244bddc1-aaf0-4e98-98c5-5302a60800a6","Type":"ContainerStarted","Data":"893412602bcc1a00f203bb4bb4a92b439624dfdec58a8a3d46866403e3c633a1"} Jul 30 22:48:11.301540 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:11.301398 2554 generic.go:356] "Generic (PLEG): container finished" podID="9ff92d75-8091-4a72-a19b-6a39fef8f5d9" containerID="8193f8a938c9cff375c2888e4ea6f7a0fdbacea8bad6a890713c969fcb3500bf" exitCode=0 Jul 30 22:48:11.301540 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:11.301485 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerDied","Data":"8193f8a938c9cff375c2888e4ea6f7a0fdbacea8bad6a890713c969fcb3500bf"} Jul 30 22:48:12.174282 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.174025 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:12.175270 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.174671 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:12.175270 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.174240 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:12.175270 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.174848 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 22:48:12.175270 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.174860 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b7cb797d-rmcrv: secret "image-registry-tls" not found Jul 30 22:48:12.175270 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.174886 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls podName:90fd9a18-a33a-45c5-b685-20e709a56cf7 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:16.174865532 +0000 UTC m=+42.647329534 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-vh474" (UID: "90fd9a18-a33a-45c5-b685-20e709a56cf7") : secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:12.175270 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.174904 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls podName:b4b22577-b8ad-4dd9-88a1-365e5bf69c27 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:16.174895124 +0000 UTC m=+42.647359115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls") pod "image-registry-64b7cb797d-rmcrv" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27") : secret "image-registry-tls" not found Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.275281 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.275408 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.275456 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.275486 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.275725 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.275804 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:16.27578449 +0000 UTC m=+42.748248484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : secret "router-metrics-certs-default" not found Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.276214 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:16.276196318 +0000 UTC m=+42.748660310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : configmap references non-existent config key: service-ca.crt Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.276284 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.276314 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls podName:4bef5592-9ae3-4e19-9d0e-b309bceb1f59 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:16.276303462 +0000 UTC m=+42.748767457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls") pod "dns-default-fg66p" (UID: "4bef5592-9ae3-4e19-9d0e-b309bceb1f59") : secret "dns-default-metrics-tls" not found Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.276365 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 22:48:12.276473 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:12.276394 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls podName:515ff8e0-d3c7-41b5-8f17-5acd0df58061 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:16.276384654 +0000 UTC m=+42.748848650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9csh6" (UID: "515ff8e0-d3c7-41b5-8f17-5acd0df58061") : secret "samples-operator-tls" not found Jul 30 22:48:12.315892 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.314880 2554 generic.go:356] "Generic (PLEG): container finished" podID="9ff92d75-8091-4a72-a19b-6a39fef8f5d9" containerID="23e16cd7709a7e64800bd4f0a6782e35afa2f14199beb0893eb0c5a7e59842a9" exitCode=0 Jul 30 22:48:12.315892 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.314940 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerDied","Data":"23e16cd7709a7e64800bd4f0a6782e35afa2f14199beb0893eb0c5a7e59842a9"} Jul 30 22:48:12.569556 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.569468 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:12.640837 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:12.640803 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:13.539713 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:13.539632 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:13.639996 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:13.639961 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:14.623638 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:14.623593 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:14.762098 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:14.762065 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:14.841189 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:14.841135 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:15.080218 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.080183 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6"] Jul 30 22:48:15.114995 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.114960 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6"] Jul 30 22:48:15.115182 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.115110 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:15.118134 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.118103 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 30 22:48:15.118329 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.118308 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2mjtk\"" Jul 30 22:48:15.119059 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.119038 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 30 22:48:15.201974 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.201927 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2bd51092-0adc-481e-bc30-83d4604f2513-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:15.202158 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.202039 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:15.242847 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.242777 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:15.302997 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.302957 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2bd51092-0adc-481e-bc30-83d4604f2513-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:15.303187 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.303070 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:15.303257 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:15.303237 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 22:48:15.303322 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:15.303312 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert podName:2bd51092-0adc-481e-bc30-83d4604f2513 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:15.803294673 +0000 UTC m=+42.275758663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-zjbk6" (UID: "2bd51092-0adc-481e-bc30-83d4604f2513") : secret "networking-console-plugin-cert" not found Jul 30 22:48:15.303662 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.303632 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2bd51092-0adc-481e-bc30-83d4604f2513-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:15.340615 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.340501 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" limit=1024 Jul 30 22:48:15.808470 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:15.808430 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:15.808912 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:15.808593 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 22:48:15.808912 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:15.808670 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert podName:2bd51092-0adc-481e-bc30-83d4604f2513 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:16.808653473 +0000 UTC m=+43.281117466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-zjbk6" (UID: "2bd51092-0adc-481e-bc30-83d4604f2513") : secret "networking-console-plugin-cert" not found Jul 30 22:48:16.212593 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:16.212549 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:16.212800 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:16.212650 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:16.212800 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.212693 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:16.212800 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.212777 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls podName:90fd9a18-a33a-45c5-b685-20e709a56cf7 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:24.212760845 +0000 UTC m=+50.685225046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-vh474" (UID: "90fd9a18-a33a-45c5-b685-20e709a56cf7") : secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:16.212931 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.212815 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 22:48:16.212931 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.212836 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b7cb797d-rmcrv: secret "image-registry-tls" not found Jul 30 22:48:16.212931 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.212899 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls podName:b4b22577-b8ad-4dd9-88a1-365e5bf69c27 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:24.212882384 +0000 UTC m=+50.685346381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls") pod "image-registry-64b7cb797d-rmcrv" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27") : secret "image-registry-tls" not found Jul 30 22:48:16.313791 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:16.313752 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:16.313951 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:16.313804 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:16.313951 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:16.313851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:16.313951 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.313863 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 22:48:16.313951 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.313928 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls podName:515ff8e0-d3c7-41b5-8f17-5acd0df58061 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:24.313912957 +0000 UTC m=+50.786376947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9csh6" (UID: "515ff8e0-d3c7-41b5-8f17-5acd0df58061") : secret "samples-operator-tls" not found Jul 30 22:48:16.314151 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.313962 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 22:48:16.314151 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:16.313972 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:16.314151 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.313995 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:24.313980843 +0000 UTC m=+50.786444832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : configmap references non-existent config key: service-ca.crt Jul 30 22:48:16.314151 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.314022 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 22:48:16.314151 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.314025 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:24.314018565 +0000 UTC m=+50.786482555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : secret "router-metrics-certs-default" not found Jul 30 22:48:16.314151 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.314046 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls podName:4bef5592-9ae3-4e19-9d0e-b309bceb1f59 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:24.314038342 +0000 UTC m=+50.786502331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls") pod "dns-default-fg66p" (UID: "4bef5592-9ae3-4e19-9d0e-b309bceb1f59") : secret "dns-default-metrics-tls" not found Jul 30 22:48:16.818685 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:16.818649 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:16.819109 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.818822 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 22:48:16.819109 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:16.818905 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert podName:2bd51092-0adc-481e-bc30-83d4604f2513 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:18.818883597 +0000 UTC m=+45.291347602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-zjbk6" (UID: "2bd51092-0adc-481e-bc30-83d4604f2513") : secret "networking-console-plugin-cert" not found Jul 30 22:48:18.334900 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.334306 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" event={"ID":"161408ab-5872-4ef0-a41c-93bf201f1e3c","Type":"ContainerStarted","Data":"babc016a3d6a1ec7f726f5608b4a1c4cf45dd290d77db30ff4d61b0b3f124470"} Jul 30 22:48:18.337140 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.336530 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw" event={"ID":"244bddc1-aaf0-4e98-98c5-5302a60800a6","Type":"ContainerStarted","Data":"010482a55ec51f2112f07ad150e5c2f3d15434fcfa4e6b1700c12245036ade1b"} Jul 30 22:48:18.339628 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.339081 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/0.log" Jul 30 22:48:18.339628 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.339114 2554 generic.go:356] "Generic (PLEG): container finished" podID="5ba57d39-8636-40b7-b529-906527b1c207" containerID="ed2f918fc628d2c2113651affc6d51dd1bdbeb5c1752b8a94a24898361395307" exitCode=255 Jul 30 22:48:18.339628 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.339315 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" event={"ID":"5ba57d39-8636-40b7-b529-906527b1c207","Type":"ContainerDied","Data":"ed2f918fc628d2c2113651affc6d51dd1bdbeb5c1752b8a94a24898361395307"} Jul 30 22:48:18.339628 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.339406 2554 scope.go:117] "RemoveContainer" containerID="ed2f918fc628d2c2113651affc6d51dd1bdbeb5c1752b8a94a24898361395307" Jul 30 22:48:18.344276 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.343957 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq" event={"ID":"93654742-1acd-4b8c-97ca-1cf1d5c73570","Type":"ContainerStarted","Data":"4aadb4604dd9c94a97a6bd11431deef9ae0a9ab1b73edad1f1b772fe172d6920"} Jul 30 22:48:18.349217 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.349195 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" event={"ID":"9ff92d75-8091-4a72-a19b-6a39fef8f5d9","Type":"ContainerStarted","Data":"a699bdae28903fd2b61ddab5f92d1136c03e8fcd1e6e928630c08e24598469f0"} Jul 30 22:48:18.351126 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.351105 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" event={"ID":"05000c78-03e9-4364-86c2-5aa37dd2dd14","Type":"ContainerStarted","Data":"16b12773ebf2954405426bc65810e4d939b5b7a5b0a439e5906cea698cbf4ecb"} Jul 30 22:48:18.361089 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.361061 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-bzz2p" event={"ID":"fce2a95b-b468-4a4d-bf2f-76e040945724","Type":"ContainerStarted","Data":"99f8335a662dc5fdf6ffd5918057780b0f8f776c6c93acd353cd8ea70a80be60"} Jul 30 22:48:18.362829 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.362808 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d8qbt" event={"ID":"1d8434f7-e903-4268-aae4-fe78c1ab339e","Type":"ContainerStarted","Data":"309b5869ffa4b7110dbd39a9c12403828484f27c0d66f64c2f1f784b913e3cda"} Jul 30 22:48:18.393215 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.393161 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bfnwj" podStartSLOduration=14.607048607 podStartE2EDuration="44.393138285s" totalImagesPullingTime="29.786089678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:18.392187568 +0000 UTC m=+44.864651607" watchObservedRunningTime="2026-07-30 22:48:18.393138285 +0000 UTC m=+44.865602298" Jul 30 22:48:18.393488 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.393461 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" podStartSLOduration=27.835950894 podStartE2EDuration="35.393453032s" totalImagesPullingTime="7.557502138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:18.359421294 +0000 UTC m=+44.831885307" watchObservedRunningTime="2026-07-30 22:48:18.393453032 +0000 UTC m=+44.865917046" Jul 30 22:48:18.452326 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.452279 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-g4fxw" podStartSLOduration=25.619228327 podStartE2EDuration="33.45225868s" totalImagesPullingTime="7.833030353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:18.45094841 +0000 UTC m=+44.923412446" watchObservedRunningTime="2026-07-30 22:48:18.45225868 +0000 UTC m=+44.924722694" Jul 30 22:48:18.497495 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.497437 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" podStartSLOduration=28.943159448 podStartE2EDuration="36.497418115s" totalImagesPullingTime="7.554258667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:18.496570739 +0000 UTC m=+44.969034752" watchObservedRunningTime="2026-07-30 22:48:18.497418115 +0000 UTC m=+44.969882128" Jul 30 22:48:18.540857 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.539784 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d8qbt" podStartSLOduration=2.981750194 podStartE2EDuration="10.53976375s" totalImagesPullingTime="7.558013556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:18.539534169 +0000 UTC m=+45.011998182" watchObservedRunningTime="2026-07-30 22:48:18.53976375 +0000 UTC m=+45.012227760" Jul 30 22:48:18.540857 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.540411 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bx5lq" podStartSLOduration=28.968624628 podStartE2EDuration="36.54040355s" totalImagesPullingTime="7.571778922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:18.518989786 +0000 UTC m=+44.991453798" watchObservedRunningTime="2026-07-30 22:48:18.54040355 +0000 UTC m=+45.012867540" Jul 30 22:48:18.563159 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.562569 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-bzz2p" podStartSLOduration=29.009164992 podStartE2EDuration="36.562548098s" totalImagesPullingTime="7.553383106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:18.561563677 +0000 UTC m=+45.034027699" watchObservedRunningTime="2026-07-30 22:48:18.562548098 +0000 UTC m=+45.035012503" Jul 30 22:48:18.755580 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.755542 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:18.755773 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.755608 2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:18.841373 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:18.841341 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:18.841573 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:18.841544 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 22:48:18.841639 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:18.841629 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert podName:2bd51092-0adc-481e-bc30-83d4604f2513 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:22.841607654 +0000 UTC m=+49.314071662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-zjbk6" (UID: "2bd51092-0adc-481e-bc30-83d4604f2513") : secret "networking-console-plugin-cert" not found Jul 30 22:48:19.247264 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.247228 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6gnrz"] Jul 30 22:48:19.268209 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.268167 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6gnrz"] Jul 30 22:48:19.268364 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.268278 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.270815 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.270789 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 30 22:48:19.271201 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.271182 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 30 22:48:19.271482 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.271463 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4z2vv\"" Jul 30 22:48:19.346165 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.346131 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67d05f84-9319-4f44-8a23-57049b3834ac-crio-socket\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.346585 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.346173 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67d05f84-9319-4f44-8a23-57049b3834ac-data-volume\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.346585 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.346278 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9m5t\" (UniqueName: \"kubernetes.io/projected/67d05f84-9319-4f44-8a23-57049b3834ac-kube-api-access-x9m5t\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.346585 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.346334 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.346585 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.346366 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.368330 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.368300 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 22:48:19.368768 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.368729 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/0.log" Jul 30 22:48:19.368853 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.368790 2554 generic.go:356] "Generic (PLEG): container finished" podID="5ba57d39-8636-40b7-b529-906527b1c207" containerID="45a606772434b5d054b5cdf113bd0b34db8792bb90cff4827a8119932c112357" exitCode=255 Jul 30 22:48:19.368914 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.368895 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" event={"ID":"5ba57d39-8636-40b7-b529-906527b1c207","Type":"ContainerDied","Data":"45a606772434b5d054b5cdf113bd0b34db8792bb90cff4827a8119932c112357"} Jul 30 22:48:19.378302 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.378280 2554 scope.go:117] "RemoveContainer" containerID="ed2f918fc628d2c2113651affc6d51dd1bdbeb5c1752b8a94a24898361395307" Jul 30 22:48:19.378415 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.378395 2554 scope.go:117] "RemoveContainer" containerID="45a606772434b5d054b5cdf113bd0b34db8792bb90cff4827a8119932c112357" Jul 30 22:48:19.378582 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.378561 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-bfq27_openshift-console-operator(5ba57d39-8636-40b7-b529-906527b1c207)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" podUID="5ba57d39-8636-40b7-b529-906527b1c207" Jul 30 22:48:19.447473 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.447444 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9m5t\" (UniqueName: \"kubernetes.io/projected/67d05f84-9319-4f44-8a23-57049b3834ac-kube-api-access-x9m5t\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.447628 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.447610 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.447713 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.447694 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.447787 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.447757 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 22:48:19.448582 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.447839 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 22:48:19.448582 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.447861 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:19.947832281 +0000 UTC m=+46.420296271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : configmap "kube-rbac-proxy" not found Jul 30 22:48:19.448582 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.447913 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:19.94789541 +0000 UTC m=+46.420359411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : secret "insights-runtime-extractor-tls" not found Jul 30 22:48:19.448582 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.448103 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67d05f84-9319-4f44-8a23-57049b3834ac-crio-socket\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.448582 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.448180 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67d05f84-9319-4f44-8a23-57049b3834ac-data-volume\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.448582 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.448316 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67d05f84-9319-4f44-8a23-57049b3834ac-crio-socket\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.448582 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.448506 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67d05f84-9319-4f44-8a23-57049b3834ac-data-volume\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.457549 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.457529 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9m5t\" (UniqueName: \"kubernetes.io/projected/67d05f84-9319-4f44-8a23-57049b3834ac-kube-api-access-x9m5t\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.954409 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.954376 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.954579 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:19.954425 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:19.954579 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.954510 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 22:48:19.954579 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.954540 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 22:48:19.954579 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.954578 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:20.954563577 +0000 UTC m=+47.427027568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : configmap "kube-rbac-proxy" not found Jul 30 22:48:19.954717 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:19.954594 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:20.954587713 +0000 UTC m=+47.427051703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : secret "insights-runtime-extractor-tls" not found Jul 30 22:48:20.352787 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.352697 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4"] Jul 30 22:48:20.374551 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.374495 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 22:48:20.374963 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.374945 2554 scope.go:117] "RemoveContainer" containerID="45a606772434b5d054b5cdf113bd0b34db8792bb90cff4827a8119932c112357" Jul 30 22:48:20.375164 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:20.375142 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-bfq27_openshift-console-operator(5ba57d39-8636-40b7-b529-906527b1c207)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" podUID="5ba57d39-8636-40b7-b529-906527b1c207" Jul 30 22:48:20.380078 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.380057 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4"] Jul 30 22:48:20.380181 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.380141 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" Jul 30 22:48:20.382703 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.382678 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 30 22:48:20.382821 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.382686 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 30 22:48:20.382821 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.382757 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4w68t\"" Jul 30 22:48:20.417583 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.417556 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mm57w_bf94eccb-32e3-45b3-9329-8991e4f7d720/dns-node-resolver/0.log" Jul 30 22:48:20.458322 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.458284 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h2vw\" (UniqueName: \"kubernetes.io/projected/e9818882-6d2d-4e4d-935c-8e0de57550ec-kube-api-access-5h2vw\") pod \"migrator-68bc745d87-5lpj4\" (UID: \"e9818882-6d2d-4e4d-935c-8e0de57550ec\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" Jul 30 22:48:20.559329 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.559292 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h2vw\" (UniqueName: \"kubernetes.io/projected/e9818882-6d2d-4e4d-935c-8e0de57550ec-kube-api-access-5h2vw\") pod \"migrator-68bc745d87-5lpj4\" (UID: \"e9818882-6d2d-4e4d-935c-8e0de57550ec\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" Jul 30 22:48:20.568140 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.568110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h2vw\" (UniqueName: \"kubernetes.io/projected/e9818882-6d2d-4e4d-935c-8e0de57550ec-kube-api-access-5h2vw\") pod \"migrator-68bc745d87-5lpj4\" (UID: \"e9818882-6d2d-4e4d-935c-8e0de57550ec\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" Jul 30 22:48:20.689278 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.689247 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" Jul 30 22:48:20.835701 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.835667 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4"] Jul 30 22:48:20.838746 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:20.838699 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9818882_6d2d_4e4d_935c_8e0de57550ec.slice/crio-97f37b2be6e8c4a66a34705882eb8ac6c42331e981f82c49a20d33bb321b86a2 WatchSource:0}: Error finding container 97f37b2be6e8c4a66a34705882eb8ac6c42331e981f82c49a20d33bb321b86a2: Status 404 returned error can't find the container with id 97f37b2be6e8c4a66a34705882eb8ac6c42331e981f82c49a20d33bb321b86a2 Jul 30 22:48:20.963167 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.963079 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:20.963167 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:20.963127 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:20.963350 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:20.963210 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 22:48:20.963350 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:20.963258 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 22:48:20.963350 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:20.963280 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:22.963265053 +0000 UTC m=+49.435729044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : configmap "kube-rbac-proxy" not found Jul 30 22:48:20.963350 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:20.963298 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:22.963286489 +0000 UTC m=+49.435750479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : secret "insights-runtime-extractor-tls" not found Jul 30 22:48:21.218051 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.217967 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9srwj_f340f7c6-8994-43b9-a22a-8b7a4883518c/node-ca/0.log" Jul 30 22:48:21.378642 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.378610 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" event={"ID":"e9818882-6d2d-4e4d-935c-8e0de57550ec","Type":"ContainerStarted","Data":"97f37b2be6e8c4a66a34705882eb8ac6c42331e981f82c49a20d33bb321b86a2"} Jul 30 22:48:21.582746 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.582655 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-h6mx5"] Jul 30 22:48:21.603559 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.603534 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-h6mx5"] Jul 30 22:48:21.603745 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.603626 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.609777 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.609751 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 30 22:48:21.609920 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.609855 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 30 22:48:21.612269 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.611144 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 30 22:48:21.612269 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.611872 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 30 22:48:21.612269 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.612052 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xwpsm\"" Jul 30 22:48:21.669956 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.669923 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-signing-cabundle\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.670127 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.670082 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-signing-key\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.670198 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.670160 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjkn7\" (UniqueName: \"kubernetes.io/projected/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-kube-api-access-wjkn7\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.771002 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.770961 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjkn7\" (UniqueName: \"kubernetes.io/projected/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-kube-api-access-wjkn7\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.771164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.771080 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-signing-cabundle\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.771164 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.771136 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-signing-key\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.771763 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.771719 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-signing-cabundle\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.773551 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.773532 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-signing-key\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.780016 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.779994 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjkn7\" (UniqueName: \"kubernetes.io/projected/2dbfc180-5cd1-4da7-9a19-c501d1ed700c-kube-api-access-wjkn7\") pod \"service-ca-98c88fbb5-h6mx5\" (UID: \"2dbfc180-5cd1-4da7-9a19-c501d1ed700c\") " pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:21.914925 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:21.914834 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" Jul 30 22:48:22.018064 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:22.018037 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d8qbt_1d8434f7-e903-4268-aae4-fe78c1ab339e/serve-healthcheck-canary/0.log" Jul 30 22:48:22.077527 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:22.077493 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-h6mx5"] Jul 30 22:48:22.338525 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:22.338491 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dbfc180_5cd1_4da7_9a19_c501d1ed700c.slice/crio-c99e83f14622ddda540d57ee8ec92116367dee6d39738e2d6868fb4dc10cf28c WatchSource:0}: Error finding container c99e83f14622ddda540d57ee8ec92116367dee6d39738e2d6868fb4dc10cf28c: Status 404 returned error can't find the container with id c99e83f14622ddda540d57ee8ec92116367dee6d39738e2d6868fb4dc10cf28c Jul 30 22:48:22.382688 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:22.382660 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" event={"ID":"2dbfc180-5cd1-4da7-9a19-c501d1ed700c","Type":"ContainerStarted","Data":"c99e83f14622ddda540d57ee8ec92116367dee6d39738e2d6868fb4dc10cf28c"} Jul 30 22:48:22.880824 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:22.880724 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:22.880984 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:22.880869 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 22:48:22.880984 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:22.880934 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert podName:2bd51092-0adc-481e-bc30-83d4604f2513 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:30.880919044 +0000 UTC m=+57.353383034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-zjbk6" (UID: "2bd51092-0adc-481e-bc30-83d4604f2513") : secret "networking-console-plugin-cert" not found Jul 30 22:48:22.981386 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:22.981348 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:22.981572 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:22.981395 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:22.981572 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:22.981491 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 22:48:22.981572 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:22.981507 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 22:48:22.981572 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:22.981557 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:26.98154423 +0000 UTC m=+53.454008225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : secret "insights-runtime-extractor-tls" not found Jul 30 22:48:22.981572 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:22.981571 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:26.981565129 +0000 UTC m=+53.454029123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : configmap "kube-rbac-proxy" not found Jul 30 22:48:23.388405 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:23.388368 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" event={"ID":"2dbfc180-5cd1-4da7-9a19-c501d1ed700c","Type":"ContainerStarted","Data":"de50425688f97a971d4f87be5466c1f8692e51620ae3ae30082b9e6507a34312"} Jul 30 22:48:23.390253 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:23.390232 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" event={"ID":"e9818882-6d2d-4e4d-935c-8e0de57550ec","Type":"ContainerStarted","Data":"7dda31f4893187cecae7b612c2d4fe6275f7734bdfe268cdf7789e7611d14e71"} Jul 30 22:48:23.390355 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:23.390272 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" event={"ID":"e9818882-6d2d-4e4d-935c-8e0de57550ec","Type":"ContainerStarted","Data":"09e8ff3951d7bf6a9cbea19d1262c9bd0b30f945b3fe4f7916dfb081e8437623"} Jul 30 22:48:23.410907 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:23.410853 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-h6mx5" podStartSLOduration=2.41083583 podStartE2EDuration="2.41083583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:23.409720162 +0000 UTC m=+49.882184171" watchObservedRunningTime="2026-07-30 22:48:23.41083583 +0000 UTC m=+49.883299846" Jul 30 22:48:23.432557 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:23.432513 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-5lpj4" podStartSLOduration=1.776969315 podStartE2EDuration="3.432498647s" totalImagesPullingTime="1.655529332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:23.431974054 +0000 UTC m=+49.904438068" watchObservedRunningTime="2026-07-30 22:48:23.432498647 +0000 UTC m=+49.904962658" Jul 30 22:48:24.293442 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.293400 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:24.293619 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.293503 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:24.293619 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.293565 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:24.293619 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.293608 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 22:48:24.293766 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.293620 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b7cb797d-rmcrv: secret "image-registry-tls" not found Jul 30 22:48:24.293766 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.293639 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls podName:90fd9a18-a33a-45c5-b685-20e709a56cf7 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:40.293618648 +0000 UTC m=+66.766082661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-vh474" (UID: "90fd9a18-a33a-45c5-b685-20e709a56cf7") : secret "cluster-monitoring-operator-tls" not found Jul 30 22:48:24.293766 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.293654 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls podName:b4b22577-b8ad-4dd9-88a1-365e5bf69c27 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:40.293645251 +0000 UTC m=+66.766109241 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls") pod "image-registry-64b7cb797d-rmcrv" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27") : secret "image-registry-tls" not found Jul 30 22:48:24.394031 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.393998 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:24.394420 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.394107 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:24.394420 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.394145 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:40.394127842 +0000 UTC m=+66.866591834 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : configmap references non-existent config key: service-ca.crt Jul 30 22:48:24.394420 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.394191 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 22:48:24.394420 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.394220 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:24.394420 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.394235 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls podName:4bef5592-9ae3-4e19-9d0e-b309bceb1f59 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:40.394220853 +0000 UTC m=+66.866684860 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls") pod "dns-default-fg66p" (UID: "4bef5592-9ae3-4e19-9d0e-b309bceb1f59") : secret "dns-default-metrics-tls" not found Jul 30 22:48:24.394420 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.394265 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:24.394420 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.394359 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 22:48:24.394420 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:24.394400 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs podName:d8393df3-ae76-4d13-bbc1-1bfe1305cb87 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:40.394389011 +0000 UTC m=+66.866853007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs") pod "router-default-66f84c8f94-gds4x" (UID: "d8393df3-ae76-4d13-bbc1-1bfe1305cb87") : secret "router-metrics-certs-default" not found Jul 30 22:48:24.396582 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.396562 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/515ff8e0-d3c7-41b5-8f17-5acd0df58061-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9csh6\" (UID: \"515ff8e0-d3c7-41b5-8f17-5acd0df58061\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:24.668649 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.668615 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" Jul 30 22:48:24.804253 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:24.804227 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6"] Jul 30 22:48:25.400994 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:25.400957 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" event={"ID":"515ff8e0-d3c7-41b5-8f17-5acd0df58061","Type":"ContainerStarted","Data":"c09391e46a1067732dadd79ea20f5c4c9d0df1a9c1603122be146a5f79dfc6ae"} Jul 30 22:48:27.018338 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:27.018250 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:27.018689 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:27.018426 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:27.018689 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:27.018567 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 22:48:27.018689 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:27.018638 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls podName:67d05f84-9319-4f44-8a23-57049b3834ac nodeName:}" failed. No retries permitted until 2026-07-30 22:48:35.018620059 +0000 UTC m=+61.491084055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnrz" (UID: "67d05f84-9319-4f44-8a23-57049b3834ac") : secret "insights-runtime-extractor-tls" not found Jul 30 22:48:27.018903 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:27.018883 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67d05f84-9319-4f44-8a23-57049b3834ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:27.409370 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:27.409289 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" event={"ID":"515ff8e0-d3c7-41b5-8f17-5acd0df58061","Type":"ContainerStarted","Data":"4b27df2ea3d927af960a61001c521cb14ca0e1c8ac66f34176884579145682d7"} Jul 30 22:48:27.409370 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:27.409339 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" event={"ID":"515ff8e0-d3c7-41b5-8f17-5acd0df58061","Type":"ContainerStarted","Data":"5c4afa744224877f5a5193c10922d71f8843478539612f5ace593eba13d19485"} Jul 30 22:48:27.429257 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:27.429207 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9csh6" podStartSLOduration=43.566119973 podStartE2EDuration="45.429191109s" totalImagesPullingTime="1.863071136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:27.428347753 +0000 UTC m=+53.900811765" watchObservedRunningTime="2026-07-30 22:48:27.429191109 +0000 UTC m=+53.901655153" Jul 30 22:48:28.755031 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:28.754992 2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:28.755498 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:28.755064 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:28.755498 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:28.755483 2554 scope.go:117] "RemoveContainer" containerID="45a606772434b5d054b5cdf113bd0b34db8792bb90cff4827a8119932c112357" Jul 30 22:48:29.418376 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:29.418349 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 22:48:29.418635 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:29.418419 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" event={"ID":"5ba57d39-8636-40b7-b529-906527b1c207","Type":"ContainerStarted","Data":"a63e10b32e5915dab366ef6f0e1486d0679efd41c6ed9d8dd09222ecb43deee2"} Jul 30 22:48:29.418762 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:29.418718 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:29.436545 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:29.436496 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" podStartSLOduration=39.880101564 podStartE2EDuration="47.436482479s" totalImagesPullingTime="7.556380915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:29.435382728 +0000 UTC m=+55.907846767" watchObservedRunningTime="2026-07-30 22:48:29.436482479 +0000 UTC m=+55.908946490" Jul 30 22:48:29.474639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:29.474612 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-bfq27" Jul 30 22:48:30.959867 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:30.959829 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:30.960360 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:30.960005 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 22:48:30.960360 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:48:30.960101 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert podName:2bd51092-0adc-481e-bc30-83d4604f2513 nodeName:}" failed. No retries permitted until 2026-07-30 22:48:46.960079993 +0000 UTC m=+73.432543988 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-zjbk6" (UID: "2bd51092-0adc-481e-bc30-83d4604f2513") : secret "networking-console-plugin-cert" not found Jul 30 22:48:34.269801 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:34.269772 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mws57" Jul 30 22:48:35.099710 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:35.099670 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:35.101982 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:35.101960 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67d05f84-9319-4f44-8a23-57049b3834ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnrz\" (UID: \"67d05f84-9319-4f44-8a23-57049b3834ac\") " pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:35.183414 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:35.183381 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4z2vv\"" Jul 30 22:48:35.191021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:35.190997 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6gnrz" Jul 30 22:48:35.344261 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:35.344234 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6gnrz"] Jul 30 22:48:35.349115 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:35.347065 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67d05f84_9319_4f44_8a23_57049b3834ac.slice/crio-5ca602ae79239bafc3444367186502ce24ed78adc31f608e4bbe2a2758e68ff3 WatchSource:0}: Error finding container 5ca602ae79239bafc3444367186502ce24ed78adc31f608e4bbe2a2758e68ff3: Status 404 returned error can't find the container with id 5ca602ae79239bafc3444367186502ce24ed78adc31f608e4bbe2a2758e68ff3 Jul 30 22:48:35.439521 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:35.439493 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gnrz" event={"ID":"67d05f84-9319-4f44-8a23-57049b3834ac","Type":"ContainerStarted","Data":"1c7bc6f7ea6b910a79206dc9818a6370aa6f0af531e70948ece82a31ecc672f0"} Jul 30 22:48:35.439650 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:35.439547 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gnrz" event={"ID":"67d05f84-9319-4f44-8a23-57049b3834ac","Type":"ContainerStarted","Data":"5ca602ae79239bafc3444367186502ce24ed78adc31f608e4bbe2a2758e68ff3"} Jul 30 22:48:36.445797 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:36.445766 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gnrz" event={"ID":"67d05f84-9319-4f44-8a23-57049b3834ac","Type":"ContainerStarted","Data":"c1d1e07c630ddee7fb0ee37ba06761cbbaaa5d1d1f3adac81fc8a45befacfe21"} Jul 30 22:48:38.454570 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:38.454533 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gnrz" event={"ID":"67d05f84-9319-4f44-8a23-57049b3834ac","Type":"ContainerStarted","Data":"b09fa8cf09f2cc507dd133da4993df40570d00e6349360eceac1e01c4670f744"} Jul 30 22:48:38.473218 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:38.473172 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6gnrz" podStartSLOduration=17.29998649 podStartE2EDuration="19.473157132s" totalImagesPullingTime="2.173170642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:48:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:38.473086931 +0000 UTC m=+64.945550944" watchObservedRunningTime="2026-07-30 22:48:38.473157132 +0000 UTC m=+64.945621143" Jul 30 22:48:39.743103 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.743066 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:39.746226 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.746197 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 22:48:39.755522 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.755501 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25f75d33-3b48-4851-b5d8-5e7cd1db81ed-metrics-certs\") pod \"network-metrics-daemon-2n7px\" (UID: \"25f75d33-3b48-4851-b5d8-5e7cd1db81ed\") " pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:39.843939 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.843910 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:39.846557 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.846542 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 22:48:39.856599 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.856581 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff6d597e-b9e7-4074-b08e-0ab2234d4a3a-original-pull-secret\") pod \"global-pull-secret-syncer-spp8w\" (UID: \"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a\") " pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:39.944344 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.944312 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:39.945817 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.945800 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-spp8w" Jul 30 22:48:39.946700 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.946683 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6whhx\" (UniqueName: \"kubernetes.io/projected/a14e59aa-d437-481b-803f-1dacc98a952c-kube-api-access-6whhx\") pod \"network-check-target-z72p4\" (UID: \"a14e59aa-d437-481b-803f-1dacc98a952c\") " pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:39.954513 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.954493 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ghwvz\"" Jul 30 22:48:39.959477 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.959455 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6fgjp\"" Jul 30 22:48:39.962413 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.962397 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:39.968793 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:39.967902 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2n7px" Jul 30 22:48:40.090788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.090761 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-spp8w"] Jul 30 22:48:40.094593 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:40.094563 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6d597e_b9e7_4074_b08e_0ab2234d4a3a.slice/crio-b319e8eb637328b09031db2d9adcd1c9ced4071ccb4bf4ef059074df7b06e46e WatchSource:0}: Error finding container b319e8eb637328b09031db2d9adcd1c9ced4071ccb4bf4ef059074df7b06e46e: Status 404 returned error can't find the container with id b319e8eb637328b09031db2d9adcd1c9ced4071ccb4bf4ef059074df7b06e46e Jul 30 22:48:40.113395 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.113367 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z72p4"] Jul 30 22:48:40.116165 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:40.116132 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14e59aa_d437_481b_803f_1dacc98a952c.slice/crio-677481b647f26238d4cbb42b084561a7f42aa61c7c0c122a1009ccd6f7315800 WatchSource:0}: Error finding container 677481b647f26238d4cbb42b084561a7f42aa61c7c0c122a1009ccd6f7315800: Status 404 returned error can't find the container with id 677481b647f26238d4cbb42b084561a7f42aa61c7c0c122a1009ccd6f7315800 Jul 30 22:48:40.129418 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.129393 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2n7px"] Jul 30 22:48:40.131873 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:40.131845 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25f75d33_3b48_4851_b5d8_5e7cd1db81ed.slice/crio-5c9006169dfb8b6a5c4092d9a5ae060c80e520d1d2e485a0413016307aef5615 WatchSource:0}: Error finding container 5c9006169dfb8b6a5c4092d9a5ae060c80e520d1d2e485a0413016307aef5615: Status 404 returned error can't find the container with id 5c9006169dfb8b6a5c4092d9a5ae060c80e520d1d2e485a0413016307aef5615 Jul 30 22:48:40.349140 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.349051 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:40.349140 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.349118 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:40.351517 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.351489 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"image-registry-64b7cb797d-rmcrv\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:40.351625 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.351496 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90fd9a18-a33a-45c5-b685-20e709a56cf7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-vh474\" (UID: \"90fd9a18-a33a-45c5-b685-20e709a56cf7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:40.449476 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.449446 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:40.449664 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.449526 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:40.449664 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.449564 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:40.450172 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.450143 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-service-ca-bundle\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:40.451936 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.451914 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bef5592-9ae3-4e19-9d0e-b309bceb1f59-metrics-tls\") pod \"dns-default-fg66p\" (UID: \"4bef5592-9ae3-4e19-9d0e-b309bceb1f59\") " pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:40.452233 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.452214 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8393df3-ae76-4d13-bbc1-1bfe1305cb87-metrics-certs\") pod \"router-default-66f84c8f94-gds4x\" (UID: \"d8393df3-ae76-4d13-bbc1-1bfe1305cb87\") " pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:40.462850 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.462825 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2n7px" event={"ID":"25f75d33-3b48-4851-b5d8-5e7cd1db81ed","Type":"ContainerStarted","Data":"5c9006169dfb8b6a5c4092d9a5ae060c80e520d1d2e485a0413016307aef5615"} Jul 30 22:48:40.464287 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.464267 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z72p4" event={"ID":"a14e59aa-d437-481b-803f-1dacc98a952c","Type":"ContainerStarted","Data":"d75c414c82cd2e15a69210ee342739b05b7aa838cc5c2960cecbc8bd4301e1cf"} Jul 30 22:48:40.464380 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.464303 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z72p4" event={"ID":"a14e59aa-d437-481b-803f-1dacc98a952c","Type":"ContainerStarted","Data":"677481b647f26238d4cbb42b084561a7f42aa61c7c0c122a1009ccd6f7315800"} Jul 30 22:48:40.464380 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.464370 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:48:40.465430 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.465411 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-spp8w" event={"ID":"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a","Type":"ContainerStarted","Data":"b319e8eb637328b09031db2d9adcd1c9ced4071ccb4bf4ef059074df7b06e46e"} Jul 30 22:48:40.480196 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.480156 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z72p4" podStartSLOduration=66.480145021 podStartE2EDuration="1m6.480145021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:40.47967047 +0000 UTC m=+66.952134480" watchObservedRunningTime="2026-07-30 22:48:40.480145021 +0000 UTC m=+66.952609033" Jul 30 22:48:40.504457 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.504436 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2qk4h\"" Jul 30 22:48:40.511878 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.511862 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m4zn7\"" Jul 30 22:48:40.512868 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.512853 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" Jul 30 22:48:40.519942 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.519925 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:40.603320 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.603292 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c9nx6\"" Jul 30 22:48:40.609215 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.609193 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g5cvn\"" Jul 30 22:48:40.611293 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.611270 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:40.617717 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.617524 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:40.652653 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.651338 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474"] Jul 30 22:48:40.656472 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:40.656437 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90fd9a18_a33a_45c5_b685_20e709a56cf7.slice/crio-d39544a4001699a18abfe199472bfd33e7b80e9b437c9a91d711302dc626aa3b WatchSource:0}: Error finding container d39544a4001699a18abfe199472bfd33e7b80e9b437c9a91d711302dc626aa3b: Status 404 returned error can't find the container with id d39544a4001699a18abfe199472bfd33e7b80e9b437c9a91d711302dc626aa3b Jul 30 22:48:40.673869 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.673802 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64b7cb797d-rmcrv"] Jul 30 22:48:40.677149 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:40.677099 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b22577_b8ad_4dd9_88a1_365e5bf69c27.slice/crio-9ae6425cc2c4b72ed52ddeff52d821deecae04725f2384b07dc082a6f41377e6 WatchSource:0}: Error finding container 9ae6425cc2c4b72ed52ddeff52d821deecae04725f2384b07dc082a6f41377e6: Status 404 returned error can't find the container with id 9ae6425cc2c4b72ed52ddeff52d821deecae04725f2384b07dc082a6f41377e6 Jul 30 22:48:40.795258 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.795229 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-66f84c8f94-gds4x"] Jul 30 22:48:40.798642 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:40.798612 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8393df3_ae76_4d13_bbc1_1bfe1305cb87.slice/crio-bbcd637fef2bade3347c34c20f7df6f37ea27d667d717451b6dde239a84dd99f WatchSource:0}: Error finding container bbcd637fef2bade3347c34c20f7df6f37ea27d667d717451b6dde239a84dd99f: Status 404 returned error can't find the container with id bbcd637fef2bade3347c34c20f7df6f37ea27d667d717451b6dde239a84dd99f Jul 30 22:48:40.813266 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:40.813228 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fg66p"] Jul 30 22:48:40.816885 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:40.816847 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bef5592_9ae3_4e19_9d0e_b309bceb1f59.slice/crio-aae1c0f7c48d81e6787c0be95c1df7c938c6d9d6015f9b1389103923f778a455 WatchSource:0}: Error finding container aae1c0f7c48d81e6787c0be95c1df7c938c6d9d6015f9b1389103923f778a455: Status 404 returned error can't find the container with id aae1c0f7c48d81e6787c0be95c1df7c938c6d9d6015f9b1389103923f778a455 Jul 30 22:48:41.475833 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.475786 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" event={"ID":"b4b22577-b8ad-4dd9-88a1-365e5bf69c27","Type":"ContainerStarted","Data":"e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a"} Jul 30 22:48:41.476088 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.475944 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:48:41.476088 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.475986 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" event={"ID":"b4b22577-b8ad-4dd9-88a1-365e5bf69c27","Type":"ContainerStarted","Data":"9ae6425cc2c4b72ed52ddeff52d821deecae04725f2384b07dc082a6f41377e6"} Jul 30 22:48:41.477204 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.477175 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" event={"ID":"90fd9a18-a33a-45c5-b685-20e709a56cf7","Type":"ContainerStarted","Data":"d39544a4001699a18abfe199472bfd33e7b80e9b437c9a91d711302dc626aa3b"} Jul 30 22:48:41.478481 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.478451 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fg66p" event={"ID":"4bef5592-9ae3-4e19-9d0e-b309bceb1f59","Type":"ContainerStarted","Data":"aae1c0f7c48d81e6787c0be95c1df7c938c6d9d6015f9b1389103923f778a455"} Jul 30 22:48:41.480116 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.480057 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-66f84c8f94-gds4x" event={"ID":"d8393df3-ae76-4d13-bbc1-1bfe1305cb87","Type":"ContainerStarted","Data":"cac2e4cc90c5ed6dacce25dccbf5bbf91dfa9d29f8960d99fa676e18726f8902"} Jul 30 22:48:41.480116 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.480101 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-66f84c8f94-gds4x" event={"ID":"d8393df3-ae76-4d13-bbc1-1bfe1305cb87","Type":"ContainerStarted","Data":"bbcd637fef2bade3347c34c20f7df6f37ea27d667d717451b6dde239a84dd99f"} Jul 30 22:48:41.497531 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.497479 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" podStartSLOduration=66.497460102 podStartE2EDuration="1m6.497460102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:41.496636046 +0000 UTC m=+67.969100275" watchObservedRunningTime="2026-07-30 22:48:41.497460102 +0000 UTC m=+67.969924095" Jul 30 22:48:41.516200 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.516139 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-66f84c8f94-gds4x" podStartSLOduration=59.516117482 podStartE2EDuration="59.516117482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:41.515555589 +0000 UTC m=+67.988019626" watchObservedRunningTime="2026-07-30 22:48:41.516117482 +0000 UTC m=+67.988581497" Jul 30 22:48:41.612426 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.612350 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:41.615363 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:41.615337 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:42.488119 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.488081 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2n7px" event={"ID":"25f75d33-3b48-4851-b5d8-5e7cd1db81ed","Type":"ContainerStarted","Data":"f89beb00930d6eeef5f81c503158e6166e0c91cb00f9c84a32b57313504f1b2f"} Jul 30 22:48:42.488587 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.488176 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2n7px" event={"ID":"25f75d33-3b48-4851-b5d8-5e7cd1db81ed","Type":"ContainerStarted","Data":"80393c39f0eb5ed4b89cb345bc5bd81ddf3950938c0d37efbe58b87c01ec9d19"} Jul 30 22:48:42.488794 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.488773 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:42.490243 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.490222 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-66f84c8f94-gds4x" Jul 30 22:48:42.510625 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.510578 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2n7px" podStartSLOduration=66.970082996 podStartE2EDuration="1m8.510562704s" totalImagesPullingTime="1.540479708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:42.509402624 +0000 UTC m=+68.981866662" watchObservedRunningTime="2026-07-30 22:48:42.510562704 +0000 UTC m=+68.983026716" Jul 30 22:48:42.654235 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.653183 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n"] Jul 30 22:48:42.669400 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.669370 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc"] Jul 30 22:48:42.669552 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.669475 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" Jul 30 22:48:42.675538 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.675509 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-p6bfz\"" Jul 30 22:48:42.675785 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.675767 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 30 22:48:42.676005 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.675988 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 30 22:48:42.676191 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.676172 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 30 22:48:42.676400 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.676384 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 30 22:48:42.698931 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.698897 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd"] Jul 30 22:48:42.700163 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.700141 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.704893 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.704871 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 30 22:48:42.705192 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.705173 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 30 22:48:42.706021 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.705856 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 30 22:48:42.707031 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.706886 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 30 22:48:42.719079 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.718966 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n"] Jul 30 22:48:42.719079 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.719000 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc"] Jul 30 22:48:42.719079 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.719020 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd"] Jul 30 22:48:42.719261 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.719113 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.721762 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.721722 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 30 22:48:42.767444 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.767356 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-h9m6j"] Jul 30 22:48:42.774802 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.774771 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-ca\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.774972 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.774818 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-hub\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.774972 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.774854 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffzsc\" (UniqueName: \"kubernetes.io/projected/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-kube-api-access-ffzsc\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.774972 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.774933 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rnk6\" (UniqueName: \"kubernetes.io/projected/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-kube-api-access-5rnk6\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.775144 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.774971 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-klusterlet-config\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.775144 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.775034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.775144 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.775073 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.775144 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.775115 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-tmp\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.775393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.775195 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.775393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.775236 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/303bc116-5e6c-4b3e-b2f5-e257ac19ea4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68964bfb6b-brr6n\" (UID: \"303bc116-5e6c-4b3e-b2f5-e257ac19ea4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" Jul 30 22:48:42.775393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.775259 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkbk7\" (UniqueName: \"kubernetes.io/projected/303bc116-5e6c-4b3e-b2f5-e257ac19ea4f-kube-api-access-kkbk7\") pod \"managed-serviceaccount-addon-agent-68964bfb6b-brr6n\" (UID: \"303bc116-5e6c-4b3e-b2f5-e257ac19ea4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" Jul 30 22:48:42.786172 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.785508 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-h9m6j"] Jul 30 22:48:42.786172 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.785579 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-h9m6j" Jul 30 22:48:42.788457 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.788162 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rfg8x\"" Jul 30 22:48:42.788457 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.788307 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 30 22:48:42.788457 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.788426 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 30 22:48:42.876028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.875868 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-ca\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.876028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.875913 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-hub\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.876028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.875950 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffzsc\" (UniqueName: \"kubernetes.io/projected/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-kube-api-access-ffzsc\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.876028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.875993 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rnk6\" (UniqueName: \"kubernetes.io/projected/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-kube-api-access-5rnk6\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.876028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.876017 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-klusterlet-config\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.876392 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.876055 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.876392 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.876103 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.876392 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.876129 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-tmp\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.876392 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.876163 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.876392 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.876195 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dtct\" (UniqueName: \"kubernetes.io/projected/5095bdb3-4d29-46e0-8fe8-9f351e438b01-kube-api-access-5dtct\") pod \"downloads-77c974ffcf-h9m6j\" (UID: \"5095bdb3-4d29-46e0-8fe8-9f351e438b01\") " pod="openshift-console/downloads-77c974ffcf-h9m6j" Jul 30 22:48:42.876392 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.876227 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/303bc116-5e6c-4b3e-b2f5-e257ac19ea4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68964bfb6b-brr6n\" (UID: \"303bc116-5e6c-4b3e-b2f5-e257ac19ea4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" Jul 30 22:48:42.876392 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.876252 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkbk7\" (UniqueName: \"kubernetes.io/projected/303bc116-5e6c-4b3e-b2f5-e257ac19ea4f-kube-api-access-kkbk7\") pod \"managed-serviceaccount-addon-agent-68964bfb6b-brr6n\" (UID: \"303bc116-5e6c-4b3e-b2f5-e257ac19ea4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" Jul 30 22:48:42.877788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.877095 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.877788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.877115 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-tmp\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.879477 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.879420 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-ca\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.880119 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.880069 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.880465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.880440 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-hub\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.880610 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.880592 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.880912 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.880890 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/303bc116-5e6c-4b3e-b2f5-e257ac19ea4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68964bfb6b-brr6n\" (UID: \"303bc116-5e6c-4b3e-b2f5-e257ac19ea4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" Jul 30 22:48:42.881438 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.881414 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-klusterlet-config\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.885418 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.885393 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffzsc\" (UniqueName: \"kubernetes.io/projected/bb3bc9d7-d886-46b4-8d67-0b6bca6a381f-kube-api-access-ffzsc\") pod \"klusterlet-addon-workmgr-55b8c54668-5wrwd\" (UID: \"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:42.885720 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.885701 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rnk6\" (UniqueName: \"kubernetes.io/projected/dca9bc28-ebdf-47e1-b693-2c3c7016f46c-kube-api-access-5rnk6\") pod \"cluster-proxy-proxy-agent-654b745699-5j8zc\" (UID: \"dca9bc28-ebdf-47e1-b693-2c3c7016f46c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:42.885879 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.885838 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkbk7\" (UniqueName: \"kubernetes.io/projected/303bc116-5e6c-4b3e-b2f5-e257ac19ea4f-kube-api-access-kkbk7\") pod \"managed-serviceaccount-addon-agent-68964bfb6b-brr6n\" (UID: \"303bc116-5e6c-4b3e-b2f5-e257ac19ea4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" Jul 30 22:48:42.977196 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.977158 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dtct\" (UniqueName: \"kubernetes.io/projected/5095bdb3-4d29-46e0-8fe8-9f351e438b01-kube-api-access-5dtct\") pod \"downloads-77c974ffcf-h9m6j\" (UID: \"5095bdb3-4d29-46e0-8fe8-9f351e438b01\") " pod="openshift-console/downloads-77c974ffcf-h9m6j" Jul 30 22:48:42.989264 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.989235 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dtct\" (UniqueName: \"kubernetes.io/projected/5095bdb3-4d29-46e0-8fe8-9f351e438b01-kube-api-access-5dtct\") pod \"downloads-77c974ffcf-h9m6j\" (UID: \"5095bdb3-4d29-46e0-8fe8-9f351e438b01\") " pod="openshift-console/downloads-77c974ffcf-h9m6j" Jul 30 22:48:42.996042 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:42.996016 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" Jul 30 22:48:43.015222 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:43.015142 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" Jul 30 22:48:43.042796 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:43.042768 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:43.097551 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:43.097522 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-h9m6j" Jul 30 22:48:45.720354 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:45.717911 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n"] Jul 30 22:48:45.728814 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:45.728759 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod303bc116_5e6c_4b3e_b2f5_e257ac19ea4f.slice/crio-d60324c5ff03437b2b893aa5b9d5b51fabeaf84c012a5eb00eb17332170c3b79 WatchSource:0}: Error finding container d60324c5ff03437b2b893aa5b9d5b51fabeaf84c012a5eb00eb17332170c3b79: Status 404 returned error can't find the container with id d60324c5ff03437b2b893aa5b9d5b51fabeaf84c012a5eb00eb17332170c3b79 Jul 30 22:48:45.963141 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:45.963067 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc"] Jul 30 22:48:45.965835 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:45.965640 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-h9m6j"] Jul 30 22:48:45.966808 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:45.966785 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd"] Jul 30 22:48:45.969011 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:45.968977 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5095bdb3_4d29_46e0_8fe8_9f351e438b01.slice/crio-f4c4160540ec5374f164338343d75503122b01f46c399e91d02162445be8f5cb WatchSource:0}: Error finding container f4c4160540ec5374f164338343d75503122b01f46c399e91d02162445be8f5cb: Status 404 returned error can't find the container with id f4c4160540ec5374f164338343d75503122b01f46c399e91d02162445be8f5cb Jul 30 22:48:45.970186 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:45.970169 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3bc9d7_d886_46b4_8d67_0b6bca6a381f.slice/crio-5863df321a0dad521d83ce71496fa681c373b35bf23d6f75b9dd98f2d4fb472c WatchSource:0}: Error finding container 5863df321a0dad521d83ce71496fa681c373b35bf23d6f75b9dd98f2d4fb472c: Status 404 returned error can't find the container with id 5863df321a0dad521d83ce71496fa681c373b35bf23d6f75b9dd98f2d4fb472c Jul 30 22:48:45.971539 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:45.971509 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddca9bc28_ebdf_47e1_b693_2c3c7016f46c.slice/crio-af26acb0e5b5b2e229f058916de55ced212fd85813616909ac616b38ce99a329 WatchSource:0}: Error finding container af26acb0e5b5b2e229f058916de55ced212fd85813616909ac616b38ce99a329: Status 404 returned error can't find the container with id af26acb0e5b5b2e229f058916de55ced212fd85813616909ac616b38ce99a329 Jul 30 22:48:46.505151 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.505108 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-spp8w" event={"ID":"ff6d597e-b9e7-4074-b08e-0ab2234d4a3a","Type":"ContainerStarted","Data":"6e3f4b3abe6d564ae06eff7aba25175ed9b7e04ed2651ad3379a87ca998f66c2"} Jul 30 22:48:46.507139 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.507110 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" event={"ID":"90fd9a18-a33a-45c5-b685-20e709a56cf7","Type":"ContainerStarted","Data":"858e66d71173918a6a2356e5cab09c5e668d6356def01ec1d3afcb3b4f7f83a0"} Jul 30 22:48:46.509202 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.509177 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fg66p" event={"ID":"4bef5592-9ae3-4e19-9d0e-b309bceb1f59","Type":"ContainerStarted","Data":"79c42987ee292069a771e65bca5f8a5f700ea8a64c2e9531b1e673c629b47982"} Jul 30 22:48:46.509390 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.509371 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fg66p" event={"ID":"4bef5592-9ae3-4e19-9d0e-b309bceb1f59","Type":"ContainerStarted","Data":"09ee1ebd6cc7fbf7bce769e627c3d3a0ea7bb80396280b2c11565e70a2c13e3f"} Jul 30 22:48:46.509519 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.509505 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:46.510442 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.510424 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-h9m6j" event={"ID":"5095bdb3-4d29-46e0-8fe8-9f351e438b01","Type":"ContainerStarted","Data":"f4c4160540ec5374f164338343d75503122b01f46c399e91d02162445be8f5cb"} Jul 30 22:48:46.511650 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.511627 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" event={"ID":"303bc116-5e6c-4b3e-b2f5-e257ac19ea4f","Type":"ContainerStarted","Data":"d60324c5ff03437b2b893aa5b9d5b51fabeaf84c012a5eb00eb17332170c3b79"} Jul 30 22:48:46.512864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.512833 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" event={"ID":"dca9bc28-ebdf-47e1-b693-2c3c7016f46c","Type":"ContainerStarted","Data":"af26acb0e5b5b2e229f058916de55ced212fd85813616909ac616b38ce99a329"} Jul 30 22:48:46.514140 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.514119 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" event={"ID":"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f","Type":"ContainerStarted","Data":"5863df321a0dad521d83ce71496fa681c373b35bf23d6f75b9dd98f2d4fb472c"} Jul 30 22:48:46.525136 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.525074 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-spp8w" podStartSLOduration=67.088175444 podStartE2EDuration="1m12.525056474s" totalImagesPullingTime="5.43688103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:46.523975364 +0000 UTC m=+72.996439377" watchObservedRunningTime="2026-07-30 22:48:46.525056474 +0000 UTC m=+72.997520484" Jul 30 22:48:46.544422 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.544361 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fg66p" podStartSLOduration=33.84000534 podStartE2EDuration="38.544339437s" totalImagesPullingTime="4.704334097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:46.542435597 +0000 UTC m=+73.014899631" watchObservedRunningTime="2026-07-30 22:48:46.544339437 +0000 UTC m=+73.016803452" Jul 30 22:48:46.562321 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:46.562271 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-vh474" podStartSLOduration=59.702777857 podStartE2EDuration="1m4.562249717s" totalImagesPullingTime="4.85947186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:46.561481676 +0000 UTC m=+73.033945688" watchObservedRunningTime="2026-07-30 22:48:46.562249717 +0000 UTC m=+73.034713732" Jul 30 22:48:47.020913 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:47.020338 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:47.023861 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:47.023833 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2bd51092-0adc-481e-bc30-83d4604f2513-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-zjbk6\" (UID: \"2bd51092-0adc-481e-bc30-83d4604f2513\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:47.228568 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:47.228533 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2mjtk\"" Jul 30 22:48:47.235951 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:47.235908 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" Jul 30 22:48:47.458168 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:47.458038 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6"] Jul 30 22:48:47.544645 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:47.544113 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" event={"ID":"2bd51092-0adc-481e-bc30-83d4604f2513","Type":"ContainerStarted","Data":"cea79104fbafe3889dcbd7293e40dd855df969d51ad393d2cf21b055ac39d27e"} Jul 30 22:48:54.581165 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.581063 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" event={"ID":"303bc116-5e6c-4b3e-b2f5-e257ac19ea4f","Type":"ContainerStarted","Data":"33d7ff41e06fd99bb959f870317e937e8cdd901e7f2852ebf52bb8befb4b4d81"} Jul 30 22:48:54.583459 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.583431 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" event={"ID":"dca9bc28-ebdf-47e1-b693-2c3c7016f46c","Type":"ContainerStarted","Data":"79a35a138fd5f517f893b9cedf0f686f7c719f794d8d640ec1a8015dbb96e11e"} Jul 30 22:48:54.583586 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.583479 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" event={"ID":"dca9bc28-ebdf-47e1-b693-2c3c7016f46c","Type":"ContainerStarted","Data":"836c8cd18e32b5878d71cfbb1e5d0112090ad446a5027018ed59647522397033"} Jul 30 22:48:54.583586 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.583498 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" event={"ID":"dca9bc28-ebdf-47e1-b693-2c3c7016f46c","Type":"ContainerStarted","Data":"e57780eb14af90aef9a48ef1900c162e07b5cad42b97390cc2980d1df206a911"} Jul 30 22:48:54.584979 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.584953 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" event={"ID":"bb3bc9d7-d886-46b4-8d67-0b6bca6a381f","Type":"ContainerStarted","Data":"cc9355e52144b3af4b7bc8457e6f48f56cd661fdc7730b60cc241fe3ce89bbe9"} Jul 30 22:48:54.585782 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.585108 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:54.587188 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.587165 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" event={"ID":"2bd51092-0adc-481e-bc30-83d4604f2513","Type":"ContainerStarted","Data":"b4793abf13f0f1e11c7616af804d8225543f00c0e050224aaf5154a180415d7b"} Jul 30 22:48:54.587338 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.587309 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" Jul 30 22:48:54.599219 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.599179 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68964bfb6b-brr6n" podStartSLOduration=4.304322375 podStartE2EDuration="12.599165044s" totalImagesPullingTime="8.294842669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:54.597385286 +0000 UTC m=+81.069849311" watchObservedRunningTime="2026-07-30 22:48:54.599165044 +0000 UTC m=+81.071629055" Jul 30 22:48:54.615361 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.615110 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b8c54668-5wrwd" podStartSLOduration=4.520274123 podStartE2EDuration="12.615093855s" totalImagesPullingTime="8.094819732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:54.614757493 +0000 UTC m=+81.087221507" watchObservedRunningTime="2026-07-30 22:48:54.615093855 +0000 UTC m=+81.087557866" Jul 30 22:48:54.634851 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.634808 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b745699-5j8zc" podStartSLOduration=4.525032517 podStartE2EDuration="12.634789368s" totalImagesPullingTime="8.109756851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:54.633896581 +0000 UTC m=+81.106360597" watchObservedRunningTime="2026-07-30 22:48:54.634789368 +0000 UTC m=+81.107253382" Jul 30 22:48:54.649689 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:54.649645 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-zjbk6" podStartSLOduration=33.049292943 podStartE2EDuration="39.649630912s" totalImagesPullingTime="6.600337969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:54.649238395 +0000 UTC m=+81.121702407" watchObservedRunningTime="2026-07-30 22:48:54.649630912 +0000 UTC m=+81.122094925" Jul 30 22:48:56.546923 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.546800 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fg66p" Jul 30 22:48:56.606061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.606027 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cvzk6"] Jul 30 22:48:56.609870 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.609778 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.612424 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.612352 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 30 22:48:56.613062 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.612940 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 30 22:48:56.613158 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.613132 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 30 22:48:56.614576 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.613475 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8rdn7\"" Jul 30 22:48:56.614750 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.614719 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 30 22:48:56.727019 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.726983 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.727019 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.727025 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-sys\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.727249 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.727059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-root\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.727249 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.727082 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-wtmp\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.727249 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.727118 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34fba92e-685c-43d5-8459-a0e0fd7cfef9-metrics-client-ca\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.727249 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.727145 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-textfile\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.727249 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.727166 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-tls\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.727249 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.727212 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c4tv\" (UniqueName: \"kubernetes.io/projected/34fba92e-685c-43d5-8459-a0e0fd7cfef9-kube-api-access-8c4tv\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.727249 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.727239 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828482 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828539 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-sys\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828580 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-root\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828605 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-wtmp\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828648 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34fba92e-685c-43d5-8459-a0e0fd7cfef9-metrics-client-ca\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828673 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-textfile\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828704 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-tls\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828770 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c4tv\" (UniqueName: \"kubernetes.io/projected/34fba92e-685c-43d5-8459-a0e0fd7cfef9-kube-api-access-8c4tv\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.828902 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828798 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.829445 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.828988 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-sys\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.829445 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.829179 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.829445 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.829251 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-root\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.829593 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.829475 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34fba92e-685c-43d5-8459-a0e0fd7cfef9-metrics-client-ca\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.829973 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.829952 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-wtmp\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.830111 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.830096 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-textfile\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.831952 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.831901 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-tls\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.832867 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.832843 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34fba92e-685c-43d5-8459-a0e0fd7cfef9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.838400 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.838375 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c4tv\" (UniqueName: \"kubernetes.io/projected/34fba92e-685c-43d5-8459-a0e0fd7cfef9-kube-api-access-8c4tv\") pod \"node-exporter-cvzk6\" (UID: \"34fba92e-685c-43d5-8459-a0e0fd7cfef9\") " pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.922589 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:56.922554 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cvzk6" Jul 30 22:48:56.933623 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:48:56.933593 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34fba92e_685c_43d5_8459_a0e0fd7cfef9.slice/crio-ddbbf76bed1711378efd3920e400cde96f58b35d42064a28c2d7e50762a63b2f WatchSource:0}: Error finding container ddbbf76bed1711378efd3920e400cde96f58b35d42064a28c2d7e50762a63b2f: Status 404 returned error can't find the container with id ddbbf76bed1711378efd3920e400cde96f58b35d42064a28c2d7e50762a63b2f Jul 30 22:48:57.606002 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:48:57.605966 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvzk6" event={"ID":"34fba92e-685c-43d5-8459-a0e0fd7cfef9","Type":"ContainerStarted","Data":"ddbbf76bed1711378efd3920e400cde96f58b35d42064a28c2d7e50762a63b2f"} Jul 30 22:49:00.525713 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:00.525672 2554 patch_prober.go:28] interesting pod/image-registry-64b7cb797d-rmcrv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 30 22:49:00.526175 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:00.525753 2554 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" podUID="b4b22577-b8ad-4dd9-88a1-365e5bf69c27" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 22:49:02.493833 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:02.493796 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:49:05.646476 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:05.646421 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-h9m6j" event={"ID":"5095bdb3-4d29-46e0-8fe8-9f351e438b01","Type":"ContainerStarted","Data":"ffae0da1ad9f918e59cdcb82d5fe7b19ec45a28cfdbc5bfda776d4c908744bf1"} Jul 30 22:49:05.646969 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:05.646556 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-h9m6j" Jul 30 22:49:05.649393 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:05.649362 2554 generic.go:356] "Generic (PLEG): container finished" podID="34fba92e-685c-43d5-8459-a0e0fd7cfef9" containerID="c429d877d39bbd4a358336a1dd04971c820f8e581ae0bff3ddf040a71a9c3bc6" exitCode=0 Jul 30 22:49:05.649512 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:05.649449 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvzk6" event={"ID":"34fba92e-685c-43d5-8459-a0e0fd7cfef9","Type":"ContainerDied","Data":"c429d877d39bbd4a358336a1dd04971c820f8e581ae0bff3ddf040a71a9c3bc6"} Jul 30 22:49:05.650069 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:05.650039 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64b7cb797d-rmcrv"] Jul 30 22:49:05.661334 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:05.661310 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-h9m6j" Jul 30 22:49:05.664314 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:05.664217 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-h9m6j" podStartSLOduration=4.531639102 podStartE2EDuration="23.664199373s" totalImagesPullingTime="19.132560271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:49:05.663554879 +0000 UTC m=+92.136018891" watchObservedRunningTime="2026-07-30 22:49:05.664199373 +0000 UTC m=+92.136663388" Jul 30 22:49:06.657290 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:06.657252 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvzk6" event={"ID":"34fba92e-685c-43d5-8459-a0e0fd7cfef9","Type":"ContainerStarted","Data":"39bbc84a95c89bddf087ffca88ce0b850cb1f2b01b2eafb4488850facc9fa184"} Jul 30 22:49:06.657893 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:06.657308 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvzk6" event={"ID":"34fba92e-685c-43d5-8459-a0e0fd7cfef9","Type":"ContainerStarted","Data":"87b0dffa5ec63b109e5683812cd840aa5a80ea26b8c463844c87063de046c4d2"} Jul 30 22:49:06.677777 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:06.677701 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cvzk6" podStartSLOduration=2.455463381 podStartE2EDuration="10.677680172s" totalImagesPullingTime="8.149179895s" totalInitContainerRuntime="73.036896ms" isStatefulPod=true podCreationTimestamp="2026-07-30 22:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:49:06.677401531 +0000 UTC m=+93.149865544" watchObservedRunningTime="2026-07-30 22:49:06.677680172 +0000 UTC m=+93.150144185" Jul 30 22:49:11.482866 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:11.482823 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z72p4" Jul 30 22:49:23.731490 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:23.731450 2554 generic.go:356] "Generic (PLEG): container finished" podID="161408ab-5872-4ef0-a41c-93bf201f1e3c" containerID="babc016a3d6a1ec7f726f5608b4a1c4cf45dd290d77db30ff4d61b0b3f124470" exitCode=0 Jul 30 22:49:23.731912 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:23.731526 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" event={"ID":"161408ab-5872-4ef0-a41c-93bf201f1e3c","Type":"ContainerDied","Data":"babc016a3d6a1ec7f726f5608b4a1c4cf45dd290d77db30ff4d61b0b3f124470"} Jul 30 22:49:23.731912 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:23.731889 2554 scope.go:117] "RemoveContainer" containerID="babc016a3d6a1ec7f726f5608b4a1c4cf45dd290d77db30ff4d61b0b3f124470" Jul 30 22:49:24.737116 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:24.737080 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-r5xsv" event={"ID":"161408ab-5872-4ef0-a41c-93bf201f1e3c","Type":"ContainerStarted","Data":"3346034ae6af9689028927f6cc59ed2afeff7536e709013ae88ab3a4c5606b51"} Jul 30 22:49:30.678103 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:30.678037 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" podUID="b4b22577-b8ad-4dd9-88a1-365e5bf69c27" containerName="registry" containerID="cri-o://e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a" gracePeriod=30 Jul 30 22:49:30.966562 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:30.966526 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:49:31.030955 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.030919 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-certificates\") pod \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " Jul 30 22:49:31.031117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.030993 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsckz\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-kube-api-access-zsckz\") pod \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " Jul 30 22:49:31.031117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031031 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-ca-trust-extracted\") pod \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " Jul 30 22:49:31.031117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031052 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-trusted-ca\") pod \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " Jul 30 22:49:31.031117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031075 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-image-registry-private-configuration\") pod \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " Jul 30 22:49:31.031117 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031096 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-installation-pull-secrets\") pod \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " Jul 30 22:49:31.031350 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031132 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-bound-sa-token\") pod \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " Jul 30 22:49:31.031350 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031160 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") pod \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\" (UID: \"b4b22577-b8ad-4dd9-88a1-365e5bf69c27\") " Jul 30 22:49:31.031350 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031192 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b4b22577-b8ad-4dd9-88a1-365e5bf69c27" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 22:49:31.031496 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031400 2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-certificates\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:49:31.031671 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.031641 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b4b22577-b8ad-4dd9-88a1-365e5bf69c27" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 22:49:31.034248 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.034223 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b4b22577-b8ad-4dd9-88a1-365e5bf69c27" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:49:31.034356 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.034239 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b4b22577-b8ad-4dd9-88a1-365e5bf69c27" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 22:49:31.034356 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.034311 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b4b22577-b8ad-4dd9-88a1-365e5bf69c27" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:49:31.034356 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.034330 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b4b22577-b8ad-4dd9-88a1-365e5bf69c27" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 22:49:31.034517 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.034375 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-kube-api-access-zsckz" (OuterVolumeSpecName: "kube-api-access-zsckz") pod "b4b22577-b8ad-4dd9-88a1-365e5bf69c27" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27"). InnerVolumeSpecName "kube-api-access-zsckz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:49:31.043083 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.043055 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b4b22577-b8ad-4dd9-88a1-365e5bf69c27" (UID: "b4b22577-b8ad-4dd9-88a1-365e5bf69c27"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 22:49:31.131850 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.131807 2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-image-registry-private-configuration\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:49:31.131850 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.131845 2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-installation-pull-secrets\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:49:31.131850 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.131857 2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-bound-sa-token\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:49:31.132078 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.131866 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-registry-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:49:31.132078 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.131874 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsckz\" (UniqueName: \"kubernetes.io/projected/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-kube-api-access-zsckz\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:49:31.132078 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.131882 2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-ca-trust-extracted\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:49:31.132078 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.131892 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b4b22577-b8ad-4dd9-88a1-365e5bf69c27-trusted-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:49:31.769788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.769752 2554 generic.go:356] "Generic (PLEG): container finished" podID="b4b22577-b8ad-4dd9-88a1-365e5bf69c27" containerID="e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a" exitCode=0 Jul 30 22:49:31.769788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.769793 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" Jul 30 22:49:31.770289 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.769826 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" event={"ID":"b4b22577-b8ad-4dd9-88a1-365e5bf69c27","Type":"ContainerDied","Data":"e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a"} Jul 30 22:49:31.770289 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.769883 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b7cb797d-rmcrv" event={"ID":"b4b22577-b8ad-4dd9-88a1-365e5bf69c27","Type":"ContainerDied","Data":"9ae6425cc2c4b72ed52ddeff52d821deecae04725f2384b07dc082a6f41377e6"} Jul 30 22:49:31.770289 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.769904 2554 scope.go:117] "RemoveContainer" containerID="e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a" Jul 30 22:49:31.778289 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.778272 2554 scope.go:117] "RemoveContainer" containerID="e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a" Jul 30 22:49:31.778524 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:49:31.778505 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a\": container with ID starting with e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a not found: ID does not exist" containerID="e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a" Jul 30 22:49:31.778575 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.778530 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a"} err="failed to get container status \"e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a\": rpc error: code = NotFound desc = could not find container \"e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a\": container with ID starting with e9c38ddcafb0c90e9678ccac451fc77862bf473f69cf884b66cc75576795e91a not found: ID does not exist" Jul 30 22:49:31.792822 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.792794 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64b7cb797d-rmcrv"] Jul 30 22:49:31.797871 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:31.797844 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64b7cb797d-rmcrv"] Jul 30 22:49:32.040354 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:32.040275 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4b22577-b8ad-4dd9-88a1-365e5bf69c27" path="/var/lib/kubelet/pods/b4b22577-b8ad-4dd9-88a1-365e5bf69c27/volumes" Jul 30 22:49:34.783085 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:34.783050 2554 generic.go:356] "Generic (PLEG): container finished" podID="fce2a95b-b468-4a4d-bf2f-76e040945724" containerID="99f8335a662dc5fdf6ffd5918057780b0f8f776c6c93acd353cd8ea70a80be60" exitCode=0 Jul 30 22:49:34.783482 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:34.783120 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-bzz2p" event={"ID":"fce2a95b-b468-4a4d-bf2f-76e040945724","Type":"ContainerDied","Data":"99f8335a662dc5fdf6ffd5918057780b0f8f776c6c93acd353cd8ea70a80be60"} Jul 30 22:49:34.783482 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:34.783457 2554 scope.go:117] "RemoveContainer" containerID="99f8335a662dc5fdf6ffd5918057780b0f8f776c6c93acd353cd8ea70a80be60" Jul 30 22:49:35.789216 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:35.789177 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-bzz2p" event={"ID":"fce2a95b-b468-4a4d-bf2f-76e040945724","Type":"ContainerStarted","Data":"a36f8ff320d0d955f7b49a462cc4c1a93299a0d2b16c2110ae9f521a835db7fa"} Jul 30 22:49:39.808991 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:39.808949 2554 generic.go:356] "Generic (PLEG): container finished" podID="05000c78-03e9-4364-86c2-5aa37dd2dd14" containerID="16b12773ebf2954405426bc65810e4d939b5b7a5b0a439e5906cea698cbf4ecb" exitCode=0 Jul 30 22:49:39.809502 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:39.809442 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" event={"ID":"05000c78-03e9-4364-86c2-5aa37dd2dd14","Type":"ContainerDied","Data":"16b12773ebf2954405426bc65810e4d939b5b7a5b0a439e5906cea698cbf4ecb"} Jul 30 22:49:39.810149 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:39.810128 2554 scope.go:117] "RemoveContainer" containerID="16b12773ebf2954405426bc65810e4d939b5b7a5b0a439e5906cea698cbf4ecb" Jul 30 22:49:40.814746 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:49:40.814709 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bcnkp" event={"ID":"05000c78-03e9-4364-86c2-5aa37dd2dd14","Type":"ContainerStarted","Data":"dc63a1c8fd6ec5a866902cdee6d8eef9f11e41d422d5fc2442a2557c12e39bda"} Jul 30 22:52:33.935955 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:33.935917 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 22:52:33.941484 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:33.941458 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 22:52:33.950624 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:33.950604 2554 kubelet.go:1648] "Image garbage collection succeeded" Jul 30 22:52:48.667331 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.667294 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p"] Jul 30 22:52:48.668930 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.667723 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4b22577-b8ad-4dd9-88a1-365e5bf69c27" containerName="registry" Jul 30 22:52:48.668930 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.667760 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4b22577-b8ad-4dd9-88a1-365e5bf69c27" containerName="registry" Jul 30 22:52:48.668930 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.667832 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4b22577-b8ad-4dd9-88a1-365e5bf69c27" containerName="registry" Jul 30 22:52:48.669775 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.669756 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.678239 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.678221 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 30 22:52:48.678585 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.678565 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rmz5j\"" Jul 30 22:52:48.679096 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.679077 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 30 22:52:48.679191 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.679097 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 30 22:52:48.679890 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.679876 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 30 22:52:48.686862 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.686840 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.686955 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.686874 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9trn\" (UniqueName: \"kubernetes.io/projected/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-kube-api-access-h9trn\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.686955 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.686916 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-manifests\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.687045 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.686973 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.687045 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.686995 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-tmp\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.713686 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.713663 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p"] Jul 30 22:52:48.788001 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.787971 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.788161 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.788021 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-tmp\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.788161 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.788073 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.788161 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.788103 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9trn\" (UniqueName: \"kubernetes.io/projected/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-kube-api-access-h9trn\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.788161 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.788146 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-manifests\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.788525 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.788493 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-manifests\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.790399 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.790370 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-tmp\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.790571 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.790553 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.790629 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.790582 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.797948 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.797930 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9trn\" (UniqueName: \"kubernetes.io/projected/78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61-kube-api-access-h9trn\") pod \"opendatahub-operator-controller-manager-5c6bb66584-dgr2p\" (UID: \"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61\") " pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:48.979681 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:48.979599 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:49.115854 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:49.115825 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p"] Jul 30 22:52:49.118682 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:52:49.118645 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a52fdd_4a4e_43cf_bbeb_d43fd5b2ea61.slice/crio-bdb8a23a76937d76d4db1ba78e2f8e470cdc8e00da68e3293f80396ff8affeff WatchSource:0}: Error finding container bdb8a23a76937d76d4db1ba78e2f8e470cdc8e00da68e3293f80396ff8affeff: Status 404 returned error can't find the container with id bdb8a23a76937d76d4db1ba78e2f8e470cdc8e00da68e3293f80396ff8affeff Jul 30 22:52:49.120491 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:49.120468 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 22:52:49.527433 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:49.527403 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" event={"ID":"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61","Type":"ContainerStarted","Data":"bdb8a23a76937d76d4db1ba78e2f8e470cdc8e00da68e3293f80396ff8affeff"} Jul 30 22:52:52.543979 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:52.543927 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" event={"ID":"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61","Type":"ContainerStarted","Data":"1cc3d963b133cdaae43ea562e336b9a90b6244203918394d17f6acbdfd7f1e79"} Jul 30 22:52:53.550047 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:53.550015 2554 generic.go:356] "Generic (PLEG): container finished" podID="78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61" containerID="1cc3d963b133cdaae43ea562e336b9a90b6244203918394d17f6acbdfd7f1e79" exitCode=0 Jul 30 22:52:53.550422 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:53.550077 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" event={"ID":"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61","Type":"ContainerDied","Data":"1cc3d963b133cdaae43ea562e336b9a90b6244203918394d17f6acbdfd7f1e79"} Jul 30 22:52:54.556206 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:54.556174 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" event={"ID":"78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61","Type":"ContainerStarted","Data":"b0fc18f8b93a7214cdf193923a921e31c300e825d93b0f6293610126b2b16a73"} Jul 30 22:52:54.556574 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:54.556270 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:52:54.581136 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:52:54.581092 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" podStartSLOduration=2.309540146 podStartE2EDuration="6.581079369s" totalImagesPullingTime="3.245449822s" totalInitContainerRuntime="1.026089401s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:52:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:52:54.578431757 +0000 UTC m=+321.050895790" watchObservedRunningTime="2026-07-30 22:52:54.581079369 +0000 UTC m=+321.053543429" Jul 30 22:53:05.563176 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:05.563146 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c6bb66584-dgr2p" Jul 30 22:53:08.476309 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.476270 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-556f7c5984-hk98v"] Jul 30 22:53:08.486089 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.486067 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.489342 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.489320 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 30 22:53:08.489478 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.489344 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 30 22:53:08.489478 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.489346 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bvs8z\"" Jul 30 22:53:08.506399 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.506373 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-556f7c5984-hk98v"] Jul 30 22:53:08.548769 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.548720 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ab46eeb-d12d-451c-94a1-b5bd521e5680-tmp\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.549031 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.549010 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl4r9\" (UniqueName: \"kubernetes.io/projected/3ab46eeb-d12d-451c-94a1-b5bd521e5680-kube-api-access-dl4r9\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.549154 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.549140 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ab46eeb-d12d-451c-94a1-b5bd521e5680-tls-certs\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.650399 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.650364 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ab46eeb-d12d-451c-94a1-b5bd521e5680-tmp\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.650565 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.650446 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl4r9\" (UniqueName: \"kubernetes.io/projected/3ab46eeb-d12d-451c-94a1-b5bd521e5680-kube-api-access-dl4r9\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.650565 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.650474 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ab46eeb-d12d-451c-94a1-b5bd521e5680-tls-certs\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.652776 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.652724 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ab46eeb-d12d-451c-94a1-b5bd521e5680-tmp\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.652886 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.652850 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ab46eeb-d12d-451c-94a1-b5bd521e5680-tls-certs\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.666297 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.666277 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl4r9\" (UniqueName: \"kubernetes.io/projected/3ab46eeb-d12d-451c-94a1-b5bd521e5680-kube-api-access-dl4r9\") pod \"kube-auth-proxy-556f7c5984-hk98v\" (UID: \"3ab46eeb-d12d-451c-94a1-b5bd521e5680\") " pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.758076 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.757999 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lz69s"] Jul 30 22:53:08.761354 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.761335 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:08.764640 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.764615 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 30 22:53:08.764718 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.764618 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-v8bwz\"" Jul 30 22:53:08.772945 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.772922 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lz69s"] Jul 30 22:53:08.795658 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.795636 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" Jul 30 22:53:08.852879 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.852771 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z9zl\" (UniqueName: \"kubernetes.io/projected/cc1d56d4-e784-44ad-af2e-a6865ff55437-kube-api-access-2z9zl\") pod \"odh-model-controller-7998c67dc4-lz69s\" (UID: \"cc1d56d4-e784-44ad-af2e-a6865ff55437\") " pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:08.852879 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.852852 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc1d56d4-e784-44ad-af2e-a6865ff55437-cert\") pod \"odh-model-controller-7998c67dc4-lz69s\" (UID: \"cc1d56d4-e784-44ad-af2e-a6865ff55437\") " pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:08.948934 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.948901 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-556f7c5984-hk98v"] Jul 30 22:53:08.951543 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:53:08.951516 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab46eeb_d12d_451c_94a1_b5bd521e5680.slice/crio-d55fd9c86be6caea76ec139fb3bf07b6a9b28d12fe8f88fdd04b25a2fda66827 WatchSource:0}: Error finding container d55fd9c86be6caea76ec139fb3bf07b6a9b28d12fe8f88fdd04b25a2fda66827: Status 404 returned error can't find the container with id d55fd9c86be6caea76ec139fb3bf07b6a9b28d12fe8f88fdd04b25a2fda66827 Jul 30 22:53:08.953307 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.953284 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z9zl\" (UniqueName: \"kubernetes.io/projected/cc1d56d4-e784-44ad-af2e-a6865ff55437-kube-api-access-2z9zl\") pod \"odh-model-controller-7998c67dc4-lz69s\" (UID: \"cc1d56d4-e784-44ad-af2e-a6865ff55437\") " pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:08.953400 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.953383 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc1d56d4-e784-44ad-af2e-a6865ff55437-cert\") pod \"odh-model-controller-7998c67dc4-lz69s\" (UID: \"cc1d56d4-e784-44ad-af2e-a6865ff55437\") " pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:08.953519 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:53:08.953508 2554 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 30 22:53:08.953572 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:53:08.953561 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc1d56d4-e784-44ad-af2e-a6865ff55437-cert podName:cc1d56d4-e784-44ad-af2e-a6865ff55437 nodeName:}" failed. No retries permitted until 2026-07-30 22:53:09.453542561 +0000 UTC m=+335.926006551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc1d56d4-e784-44ad-af2e-a6865ff55437-cert") pod "odh-model-controller-7998c67dc4-lz69s" (UID: "cc1d56d4-e784-44ad-af2e-a6865ff55437") : secret "odh-model-controller-webhook-cert" not found Jul 30 22:53:08.963729 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:08.963704 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z9zl\" (UniqueName: \"kubernetes.io/projected/cc1d56d4-e784-44ad-af2e-a6865ff55437-kube-api-access-2z9zl\") pod \"odh-model-controller-7998c67dc4-lz69s\" (UID: \"cc1d56d4-e784-44ad-af2e-a6865ff55437\") " pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:09.457036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.456996 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc1d56d4-e784-44ad-af2e-a6865ff55437-cert\") pod \"odh-model-controller-7998c67dc4-lz69s\" (UID: \"cc1d56d4-e784-44ad-af2e-a6865ff55437\") " pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:09.459578 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.459557 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc1d56d4-e784-44ad-af2e-a6865ff55437-cert\") pod \"odh-model-controller-7998c67dc4-lz69s\" (UID: \"cc1d56d4-e784-44ad-af2e-a6865ff55437\") " pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:09.589893 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.589860 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks"] Jul 30 22:53:09.594143 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.594116 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.596588 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.596568 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 30 22:53:09.596588 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.596573 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 30 22:53:09.596873 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.596857 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 30 22:53:09.596928 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.596859 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wmbpb\"" Jul 30 22:53:09.596928 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.596860 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 30 22:53:09.606773 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.606752 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks"] Jul 30 22:53:09.620825 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.620762 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" event={"ID":"3ab46eeb-d12d-451c-94a1-b5bd521e5680","Type":"ContainerStarted","Data":"d55fd9c86be6caea76ec139fb3bf07b6a9b28d12fe8f88fdd04b25a2fda66827"} Jul 30 22:53:09.658507 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.658467 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.658507 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.658506 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.658796 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.658573 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.658796 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.658598 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.658796 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.658628 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.658796 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.658663 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rjws\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-kube-api-access-2rjws\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.658796 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.658778 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.672495 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.672463 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:09.759977 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.759231 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.759977 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.759275 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.759977 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.759304 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.759977 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.759341 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rjws\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-kube-api-access-2rjws\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.759977 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.759376 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.759977 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.759448 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.759977 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.759474 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.760956 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.760928 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.765524 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.764236 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.765524 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.764781 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.765524 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.765206 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.765524 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.765464 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.781560 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.781499 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.782216 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.782176 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rjws\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-kube-api-access-2rjws\") pod \"istiod-openshift-gateway-7b7995d5cd-th9ks\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:09.846649 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.846143 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lz69s"] Jul 30 22:53:09.905752 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:09.905275 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:10.074191 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:10.074090 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks"] Jul 30 22:53:10.076518 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:53:10.076485 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode620cbbd_5a1e_4d6d_9ddb_5c026baa74a9.slice/crio-6d7a2c4cb956a8fd92c791a5692886982856f36f9ce2ea7b7fb5f8d370af99b0 WatchSource:0}: Error finding container 6d7a2c4cb956a8fd92c791a5692886982856f36f9ce2ea7b7fb5f8d370af99b0: Status 404 returned error can't find the container with id 6d7a2c4cb956a8fd92c791a5692886982856f36f9ce2ea7b7fb5f8d370af99b0 Jul 30 22:53:10.628502 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:10.628460 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" event={"ID":"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9","Type":"ContainerStarted","Data":"6d7a2c4cb956a8fd92c791a5692886982856f36f9ce2ea7b7fb5f8d370af99b0"} Jul 30 22:53:10.630410 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:10.630342 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" event={"ID":"cc1d56d4-e784-44ad-af2e-a6865ff55437","Type":"ContainerStarted","Data":"51f6a237ba1ed7e5177a0f1550554d4870c014f2a6074cca30a68ba36a2bdd34"} Jul 30 22:53:14.380899 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.380853 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 30 22:53:14.381175 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.380941 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 30 22:53:14.654121 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.654009 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" event={"ID":"3ab46eeb-d12d-451c-94a1-b5bd521e5680","Type":"ContainerStarted","Data":"0e65c625665ea4deea7c67a4f043c29ec9239db99cac4436e2ac70a283879634"} Jul 30 22:53:14.656005 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.655973 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" event={"ID":"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9","Type":"ContainerStarted","Data":"f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3"} Jul 30 22:53:14.656167 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.656041 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:14.657750 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.657712 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" event={"ID":"cc1d56d4-e784-44ad-af2e-a6865ff55437","Type":"ContainerStarted","Data":"a9e3bbb1f4b3883d0ea039b956d36e1df9e402fd45a8bf2199a02d5ebfa3c193"} Jul 30 22:53:14.657906 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.657886 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:14.671610 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.671560 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-556f7c5984-hk98v" podStartSLOduration=1.258564292 podStartE2EDuration="6.671546897s" totalImagesPullingTime="5.412982605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:53:14.669770891 +0000 UTC m=+341.142234905" watchObservedRunningTime="2026-07-30 22:53:14.671546897 +0000 UTC m=+341.144010973" Jul 30 22:53:14.691558 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.691495 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" podStartSLOduration=1.38973249 podStartE2EDuration="5.691472934s" totalImagesPullingTime="4.301740444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:53:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:53:14.688724281 +0000 UTC m=+341.161188296" watchObservedRunningTime="2026-07-30 22:53:14.691472934 +0000 UTC m=+341.163936939" Jul 30 22:53:14.706651 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:14.706590 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" podStartSLOduration=2.182819972 podStartE2EDuration="6.706567384s" totalImagesPullingTime="4.523747412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:53:14.706001064 +0000 UTC m=+341.178465077" watchObservedRunningTime="2026-07-30 22:53:14.706567384 +0000 UTC m=+341.179031397" Jul 30 22:53:15.664084 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:15.664053 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:19.699934 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.699905 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl"] Jul 30 22:53:19.703530 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.703507 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:19.705881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.705854 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vcqrq\"" Jul 30 22:53:19.706041 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.706022 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 30 22:53:19.716429 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.716401 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl"] Jul 30 22:53:19.718586 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.718565 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6"] Jul 30 22:53:19.722010 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.721990 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:19.724832 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.724809 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 30 22:53:19.725229 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.725074 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-q5jbt\"" Jul 30 22:53:19.737152 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.737118 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6"] Jul 30 22:53:19.853216 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.853175 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22647def-c7dc-43d6-9329-699e1980f14b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6hmd6\" (UID: \"22647def-c7dc-43d6-9329-699e1980f14b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:19.853416 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.853236 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv7qf\" (UniqueName: \"kubernetes.io/projected/22647def-c7dc-43d6-9329-699e1980f14b-kube-api-access-zv7qf\") pod \"llmisvc-controller-manager-777c6bd7d5-6hmd6\" (UID: \"22647def-c7dc-43d6-9329-699e1980f14b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:19.853416 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.853353 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdghj\" (UniqueName: \"kubernetes.io/projected/8feacd87-042b-4901-a834-81638db155c3-kube-api-access-pdghj\") pod \"kserve-controller-manager-d6f4b65b8-w9ktl\" (UID: \"8feacd87-042b-4901-a834-81638db155c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:19.853416 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.853398 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8feacd87-042b-4901-a834-81638db155c3-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9ktl\" (UID: \"8feacd87-042b-4901-a834-81638db155c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:19.954834 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.954746 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv7qf\" (UniqueName: \"kubernetes.io/projected/22647def-c7dc-43d6-9329-699e1980f14b-kube-api-access-zv7qf\") pod \"llmisvc-controller-manager-777c6bd7d5-6hmd6\" (UID: \"22647def-c7dc-43d6-9329-699e1980f14b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:19.954834 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.954830 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdghj\" (UniqueName: \"kubernetes.io/projected/8feacd87-042b-4901-a834-81638db155c3-kube-api-access-pdghj\") pod \"kserve-controller-manager-d6f4b65b8-w9ktl\" (UID: \"8feacd87-042b-4901-a834-81638db155c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:19.955034 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.954851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8feacd87-042b-4901-a834-81638db155c3-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9ktl\" (UID: \"8feacd87-042b-4901-a834-81638db155c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:19.955034 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.954893 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22647def-c7dc-43d6-9329-699e1980f14b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6hmd6\" (UID: \"22647def-c7dc-43d6-9329-699e1980f14b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:19.955034 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:53:19.954993 2554 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 30 22:53:19.955186 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:53:19.955068 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22647def-c7dc-43d6-9329-699e1980f14b-cert podName:22647def-c7dc-43d6-9329-699e1980f14b nodeName:}" failed. No retries permitted until 2026-07-30 22:53:20.455046866 +0000 UTC m=+346.927510856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/22647def-c7dc-43d6-9329-699e1980f14b-cert") pod "llmisvc-controller-manager-777c6bd7d5-6hmd6" (UID: "22647def-c7dc-43d6-9329-699e1980f14b") : secret "llmisvc-webhook-server-cert" not found Jul 30 22:53:19.957485 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.957459 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8feacd87-042b-4901-a834-81638db155c3-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9ktl\" (UID: \"8feacd87-042b-4901-a834-81638db155c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:19.984233 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.984208 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv7qf\" (UniqueName: \"kubernetes.io/projected/22647def-c7dc-43d6-9329-699e1980f14b-kube-api-access-zv7qf\") pod \"llmisvc-controller-manager-777c6bd7d5-6hmd6\" (UID: \"22647def-c7dc-43d6-9329-699e1980f14b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:19.991234 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:19.991209 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdghj\" (UniqueName: \"kubernetes.io/projected/8feacd87-042b-4901-a834-81638db155c3-kube-api-access-pdghj\") pod \"kserve-controller-manager-d6f4b65b8-w9ktl\" (UID: \"8feacd87-042b-4901-a834-81638db155c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:20.014390 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:20.014365 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:20.149074 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:20.149041 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl"] Jul 30 22:53:20.152150 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:53:20.152121 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8feacd87_042b_4901_a834_81638db155c3.slice/crio-4d0a6af90d7e39291371740ef387a6bb4ad94898b6a7f7bee7424fb2354e3eb7 WatchSource:0}: Error finding container 4d0a6af90d7e39291371740ef387a6bb4ad94898b6a7f7bee7424fb2354e3eb7: Status 404 returned error can't find the container with id 4d0a6af90d7e39291371740ef387a6bb4ad94898b6a7f7bee7424fb2354e3eb7 Jul 30 22:53:20.460036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:20.459981 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22647def-c7dc-43d6-9329-699e1980f14b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6hmd6\" (UID: \"22647def-c7dc-43d6-9329-699e1980f14b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:20.462432 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:20.462401 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22647def-c7dc-43d6-9329-699e1980f14b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6hmd6\" (UID: \"22647def-c7dc-43d6-9329-699e1980f14b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:20.635832 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:20.635804 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:20.686571 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:20.686540 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" event={"ID":"8feacd87-042b-4901-a834-81638db155c3","Type":"ContainerStarted","Data":"4d0a6af90d7e39291371740ef387a6bb4ad94898b6a7f7bee7424fb2354e3eb7"} Jul 30 22:53:20.771921 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:20.771894 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6"] Jul 30 22:53:20.774367 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:53:20.774331 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod22647def_c7dc_43d6_9329_699e1980f14b.slice/crio-31604d9386983ac41eb8cbf17a9a31c1e6b8c16e288a97e96c48d3d96b9c0ce3 WatchSource:0}: Error finding container 31604d9386983ac41eb8cbf17a9a31c1e6b8c16e288a97e96c48d3d96b9c0ce3: Status 404 returned error can't find the container with id 31604d9386983ac41eb8cbf17a9a31c1e6b8c16e288a97e96c48d3d96b9c0ce3 Jul 30 22:53:21.692429 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:21.692385 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" event={"ID":"22647def-c7dc-43d6-9329-699e1980f14b","Type":"ContainerStarted","Data":"31604d9386983ac41eb8cbf17a9a31c1e6b8c16e288a97e96c48d3d96b9c0ce3"} Jul 30 22:53:23.704941 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:23.704913 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" event={"ID":"8feacd87-042b-4901-a834-81638db155c3","Type":"ContainerStarted","Data":"3edf5aa7816429054fe7685e245083a6db1418a73209d8c84fa76de5a4c49878"} Jul 30 22:53:23.705278 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:23.704973 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:53:23.706345 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:23.706322 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" event={"ID":"22647def-c7dc-43d6-9329-699e1980f14b","Type":"ContainerStarted","Data":"229a1454c452cbe3de1bc4ec6de5423912a7cc41fbb8507d60e7e008c31b1fe9"} Jul 30 22:53:23.706433 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:23.706414 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:23.722011 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:23.721948 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" podStartSLOduration=1.977416949 podStartE2EDuration="4.721931645s" totalImagesPullingTime="2.744514696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:53:23.721670045 +0000 UTC m=+350.194134056" watchObservedRunningTime="2026-07-30 22:53:23.721931645 +0000 UTC m=+350.194395656" Jul 30 22:53:23.739042 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:23.738981 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" podStartSLOduration=1.881816698 podStartE2EDuration="4.7389613s" totalImagesPullingTime="2.857144602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:53:23.736514155 +0000 UTC m=+350.208978187" watchObservedRunningTime="2026-07-30 22:53:23.7389613 +0000 UTC m=+350.211425313" Jul 30 22:53:25.664009 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:25.663979 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-lz69s" Jul 30 22:53:27.837600 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:27.837517 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks"] Jul 30 22:53:27.838062 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:27.837757 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" podUID="e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" containerName="discovery" containerID="cri-o://f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3" gracePeriod=30 Jul 30 22:53:28.086639 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.086616 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:28.231653 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.231612 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-token\") pod \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " Jul 30 22:53:28.231827 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.231662 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-kubeconfig\") pod \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " Jul 30 22:53:28.231827 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.231690 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rjws\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-kube-api-access-2rjws\") pod \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " Jul 30 22:53:28.231827 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.231722 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-local-certs\") pod \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " Jul 30 22:53:28.231827 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.231778 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-dns-cert\") pod \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " Jul 30 22:53:28.231827 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.231825 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-ca-configmap\") pod \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " Jul 30 22:53:28.232094 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.231862 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-cacerts\") pod \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\" (UID: \"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9\") " Jul 30 22:53:28.232275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.232239 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" (UID: "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 22:53:28.234374 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.234296 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-local-certs" (OuterVolumeSpecName: "local-certs") pod "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" (UID: "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 22:53:28.234490 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.234446 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" (UID: "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 22:53:28.234679 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.234635 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-cacerts" (OuterVolumeSpecName: "cacerts") pod "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" (UID: "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 22:53:28.234810 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.234698 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" (UID: "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:53:28.234810 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.234702 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-kube-api-access-2rjws" (OuterVolumeSpecName: "kube-api-access-2rjws") pod "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" (UID: "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9"). InnerVolumeSpecName "kube-api-access-2rjws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:53:28.235575 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.235548 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" (UID: "e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 22:53:28.333304 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.333269 2554 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-ca-configmap\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:53:28.333304 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.333298 2554 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-cacerts\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:53:28.333304 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.333308 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-token\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:53:28.333527 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.333317 2554 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-kubeconfig\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:53:28.333527 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.333325 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rjws\" (UniqueName: \"kubernetes.io/projected/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-kube-api-access-2rjws\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:53:28.333527 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.333334 2554 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-local-certs\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:53:28.333527 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.333344 2554 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9-istio-csr-dns-cert\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:53:28.733214 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.733179 2554 generic.go:356] "Generic (PLEG): container finished" podID="e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" containerID="f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3" exitCode=0 Jul 30 22:53:28.733383 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.733253 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" Jul 30 22:53:28.733383 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.733269 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" event={"ID":"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9","Type":"ContainerDied","Data":"f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3"} Jul 30 22:53:28.733500 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.733386 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks" event={"ID":"e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9","Type":"ContainerDied","Data":"6d7a2c4cb956a8fd92c791a5692886982856f36f9ce2ea7b7fb5f8d370af99b0"} Jul 30 22:53:28.733500 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.733415 2554 scope.go:117] "RemoveContainer" containerID="f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3" Jul 30 22:53:28.742936 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.742920 2554 scope.go:117] "RemoveContainer" containerID="f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3" Jul 30 22:53:28.743160 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:53:28.743142 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3\": container with ID starting with f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3 not found: ID does not exist" containerID="f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3" Jul 30 22:53:28.743207 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.743168 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3"} err="failed to get container status \"f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3\": rpc error: code = NotFound desc = could not find container \"f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3\": container with ID starting with f14308da39553142701eb6398623d9380847d3dc5ee9876d7c30a2ec0e8324b3 not found: ID does not exist" Jul 30 22:53:28.755762 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.755725 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks"] Jul 30 22:53:28.760212 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:28.760188 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-th9ks"] Jul 30 22:53:30.036678 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:30.036635 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" path="/var/lib/kubelet/pods/e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9/volumes" Jul 30 22:53:54.713278 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:54.713245 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6hmd6" Jul 30 22:53:54.716095 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:53:54.716071 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9ktl" Jul 30 22:54:23.382830 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.382794 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j"] Jul 30 22:54:23.383182 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.383148 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" containerName="discovery" Jul 30 22:54:23.383182 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.383159 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" containerName="discovery" Jul 30 22:54:23.383250 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.383221 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e620cbbd-5a1e-4d6d-9ddb-5c026baa74a9" containerName="discovery" Jul 30 22:54:23.386288 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.386273 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" Jul 30 22:54:23.395754 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.395718 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-v5wwp\"" Jul 30 22:54:23.395863 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.395842 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 30 22:54:23.396652 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.396630 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 30 22:54:23.411486 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.411462 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 30 22:54:23.413338 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.413319 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j"] Jul 30 22:54:23.496603 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.496567 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5fzw\" (UniqueName: \"kubernetes.io/projected/71f3036e-61ae-4e9f-8d35-3a274b69d054-kube-api-access-j5fzw\") pod \"dns-operator-controller-manager-5985564776-rkf9j\" (UID: \"71f3036e-61ae-4e9f-8d35-3a274b69d054\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" Jul 30 22:54:23.597773 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.597710 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5fzw\" (UniqueName: \"kubernetes.io/projected/71f3036e-61ae-4e9f-8d35-3a274b69d054-kube-api-access-j5fzw\") pod \"dns-operator-controller-manager-5985564776-rkf9j\" (UID: \"71f3036e-61ae-4e9f-8d35-3a274b69d054\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" Jul 30 22:54:23.605675 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.605648 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5fzw\" (UniqueName: \"kubernetes.io/projected/71f3036e-61ae-4e9f-8d35-3a274b69d054-kube-api-access-j5fzw\") pod \"dns-operator-controller-manager-5985564776-rkf9j\" (UID: \"71f3036e-61ae-4e9f-8d35-3a274b69d054\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" Jul 30 22:54:23.695671 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.695645 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" Jul 30 22:54:23.827560 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.827538 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j"] Jul 30 22:54:23.829845 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:54:23.829817 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f3036e_61ae_4e9f_8d35_3a274b69d054.slice/crio-c7807605745f4bca4c3f15ac57d6ecd9206bd157a3398a99f46a9acf7f26bd16 WatchSource:0}: Error finding container c7807605745f4bca4c3f15ac57d6ecd9206bd157a3398a99f46a9acf7f26bd16: Status 404 returned error can't find the container with id c7807605745f4bca4c3f15ac57d6ecd9206bd157a3398a99f46a9acf7f26bd16 Jul 30 22:54:23.979311 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:23.979244 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" event={"ID":"71f3036e-61ae-4e9f-8d35-3a274b69d054","Type":"ContainerStarted","Data":"c7807605745f4bca4c3f15ac57d6ecd9206bd157a3398a99f46a9acf7f26bd16"} Jul 30 22:54:25.368929 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:25.368886 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6"] Jul 30 22:54:25.374182 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:25.374158 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:25.376669 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:25.376641 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7mmzl\"" Jul 30 22:54:25.382493 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:25.382277 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6"] Jul 30 22:54:25.516289 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:25.516245 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xnvx\" (UniqueName: \"kubernetes.io/projected/f812deb2-68f9-4c97-8211-340b1a6a8b44-kube-api-access-8xnvx\") pod \"limitador-operator-controller-manager-6c549b6d85-vzxk6\" (UID: \"f812deb2-68f9-4c97-8211-340b1a6a8b44\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:25.617397 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:25.617360 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xnvx\" (UniqueName: \"kubernetes.io/projected/f812deb2-68f9-4c97-8211-340b1a6a8b44-kube-api-access-8xnvx\") pod \"limitador-operator-controller-manager-6c549b6d85-vzxk6\" (UID: \"f812deb2-68f9-4c97-8211-340b1a6a8b44\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:25.630081 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:25.630015 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xnvx\" (UniqueName: \"kubernetes.io/projected/f812deb2-68f9-4c97-8211-340b1a6a8b44-kube-api-access-8xnvx\") pod \"limitador-operator-controller-manager-6c549b6d85-vzxk6\" (UID: \"f812deb2-68f9-4c97-8211-340b1a6a8b44\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:25.688322 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:25.688292 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:26.494701 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:26.494676 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6"] Jul 30 22:54:26.497206 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:54:26.497182 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf812deb2_68f9_4c97_8211_340b1a6a8b44.slice/crio-55bff6a3afd20108c81517c6353cb818b113bd3696ec23196263922d2298e68e WatchSource:0}: Error finding container 55bff6a3afd20108c81517c6353cb818b113bd3696ec23196263922d2298e68e: Status 404 returned error can't find the container with id 55bff6a3afd20108c81517c6353cb818b113bd3696ec23196263922d2298e68e Jul 30 22:54:26.995955 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:26.995921 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" event={"ID":"71f3036e-61ae-4e9f-8d35-3a274b69d054","Type":"ContainerStarted","Data":"952ca85855e9bd0218e7be4ce143d3f2ff09d78622279acbc0d8e103203d7d3b"} Jul 30 22:54:26.996139 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:26.996005 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" Jul 30 22:54:26.997239 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:26.997217 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" event={"ID":"f812deb2-68f9-4c97-8211-340b1a6a8b44","Type":"ContainerStarted","Data":"55bff6a3afd20108c81517c6353cb818b113bd3696ec23196263922d2298e68e"} Jul 30 22:54:27.014160 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:27.014100 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" podStartSLOduration=1.421568986 podStartE2EDuration="4.014083214s" totalImagesPullingTime="2.592514228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:54:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:54:27.012814712 +0000 UTC m=+413.485278725" watchObservedRunningTime="2026-07-30 22:54:27.014083214 +0000 UTC m=+413.486547226" Jul 30 22:54:28.272008 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.271971 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-m8f5q"] Jul 30 22:54:28.275726 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.275698 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" Jul 30 22:54:28.279865 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.279842 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-h6jmb\"" Jul 30 22:54:28.290830 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.290806 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-m8f5q"] Jul 30 22:54:28.336532 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.336490 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zprlz\" (UniqueName: \"kubernetes.io/projected/6954eb3a-cfdb-4366-95d5-8e88dacb394d-kube-api-access-zprlz\") pod \"authorino-operator-68d88948d8-m8f5q\" (UID: \"6954eb3a-cfdb-4366-95d5-8e88dacb394d\") " pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" Jul 30 22:54:28.437053 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.437011 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zprlz\" (UniqueName: \"kubernetes.io/projected/6954eb3a-cfdb-4366-95d5-8e88dacb394d-kube-api-access-zprlz\") pod \"authorino-operator-68d88948d8-m8f5q\" (UID: \"6954eb3a-cfdb-4366-95d5-8e88dacb394d\") " pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" Jul 30 22:54:28.448592 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.448561 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zprlz\" (UniqueName: \"kubernetes.io/projected/6954eb3a-cfdb-4366-95d5-8e88dacb394d-kube-api-access-zprlz\") pod \"authorino-operator-68d88948d8-m8f5q\" (UID: \"6954eb3a-cfdb-4366-95d5-8e88dacb394d\") " pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" Jul 30 22:54:28.587327 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.587260 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" Jul 30 22:54:28.719700 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:28.719627 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-m8f5q"] Jul 30 22:54:28.721817 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:54:28.721789 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6954eb3a_cfdb_4366_95d5_8e88dacb394d.slice/crio-fba9b56449499d18d1be8cc32e5f6ab1702d7ffeef1b1dac165d6c73f9a56c82 WatchSource:0}: Error finding container fba9b56449499d18d1be8cc32e5f6ab1702d7ffeef1b1dac165d6c73f9a56c82: Status 404 returned error can't find the container with id fba9b56449499d18d1be8cc32e5f6ab1702d7ffeef1b1dac165d6c73f9a56c82 Jul 30 22:54:29.008970 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:29.008933 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" event={"ID":"6954eb3a-cfdb-4366-95d5-8e88dacb394d","Type":"ContainerStarted","Data":"fba9b56449499d18d1be8cc32e5f6ab1702d7ffeef1b1dac165d6c73f9a56c82"} Jul 30 22:54:29.010431 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:29.010405 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" event={"ID":"f812deb2-68f9-4c97-8211-340b1a6a8b44","Type":"ContainerStarted","Data":"a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550"} Jul 30 22:54:29.010567 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:29.010550 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:29.052308 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:29.052257 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" podStartSLOduration=1.956704165 podStartE2EDuration="4.052239366s" totalImagesPullingTime="2.095535201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:54:29.051457075 +0000 UTC m=+415.523921088" watchObservedRunningTime="2026-07-30 22:54:29.052239366 +0000 UTC m=+415.524703379" Jul 30 22:54:31.022921 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:31.022886 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" event={"ID":"6954eb3a-cfdb-4366-95d5-8e88dacb394d","Type":"ContainerStarted","Data":"26fb967fd743fadf26a9d4fbf55cbc38ac71e81f6c4452bbc7a82e11a6210486"} Jul 30 22:54:31.023322 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:31.023006 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" Jul 30 22:54:31.041286 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:31.041224 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" podStartSLOduration=1.4570639060000001 podStartE2EDuration="3.041205864s" totalImagesPullingTime="1.584141958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:54:31.040493172 +0000 UTC m=+417.512957184" watchObservedRunningTime="2026-07-30 22:54:31.041205864 +0000 UTC m=+417.513669876" Jul 30 22:54:38.004606 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:38.004576 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-rkf9j" Jul 30 22:54:40.018969 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.018939 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:40.658841 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.658800 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m"] Jul 30 22:54:40.662476 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.662453 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:40.665418 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.665389 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 30 22:54:40.665513 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.665421 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8bl7q\"" Jul 30 22:54:40.665849 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.665829 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 30 22:54:40.670906 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.670850 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m"] Jul 30 22:54:40.746494 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.746432 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:40.746494 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.746503 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:40.746711 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.746611 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj7db\" (UniqueName: \"kubernetes.io/projected/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-kube-api-access-sj7db\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:40.847614 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.847574 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj7db\" (UniqueName: \"kubernetes.io/projected/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-kube-api-access-sj7db\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:40.847789 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.847646 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:40.847789 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.847689 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:40.847925 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:40.847853 2554 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 30 22:54:40.847984 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:40.847971 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-plugin-serving-cert podName:37a2509b-a9d3-42eb-8ca4-8388ce6dc247 nodeName:}" failed. No retries permitted until 2026-07-30 22:54:41.347948257 +0000 UTC m=+427.820412252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-m9x2m" (UID: "37a2509b-a9d3-42eb-8ca4-8388ce6dc247") : secret "plugin-serving-cert" not found Jul 30 22:54:40.848358 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.848340 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:40.859791 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:40.859758 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj7db\" (UniqueName: \"kubernetes.io/projected/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-kube-api-access-sj7db\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:41.353422 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:41.353381 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:41.355825 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:41.355804 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a2509b-a9d3-42eb-8ca4-8388ce6dc247-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-m9x2m\" (UID: \"37a2509b-a9d3-42eb-8ca4-8388ce6dc247\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:41.578227 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:41.578190 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" Jul 30 22:54:41.717793 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:41.717765 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m"] Jul 30 22:54:41.720434 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:54:41.720402 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37a2509b_a9d3_42eb_8ca4_8388ce6dc247.slice/crio-65791eef7320149f0565a343e37a6b478bd7a974c0839e95b983c428f7bfff10 WatchSource:0}: Error finding container 65791eef7320149f0565a343e37a6b478bd7a974c0839e95b983c428f7bfff10: Status 404 returned error can't find the container with id 65791eef7320149f0565a343e37a6b478bd7a974c0839e95b983c428f7bfff10 Jul 30 22:54:42.036007 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:42.035980 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-m8f5q" Jul 30 22:54:42.077333 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:42.077299 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" event={"ID":"37a2509b-a9d3-42eb-8ca4-8388ce6dc247","Type":"ContainerStarted","Data":"65791eef7320149f0565a343e37a6b478bd7a974c0839e95b983c428f7bfff10"} Jul 30 22:54:47.103699 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:47.103662 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" event={"ID":"37a2509b-a9d3-42eb-8ca4-8388ce6dc247","Type":"ContainerStarted","Data":"62e6e21dd36e8ca0804110f1336ad9f8924e6684bdad0061ef00c61b44e85973"} Jul 30 22:54:47.124943 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:47.124885 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-m9x2m" podStartSLOduration=2.407692446 podStartE2EDuration="7.124869775s" totalImagesPullingTime="4.717177329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:54:47.123260766 +0000 UTC m=+433.595724782" watchObservedRunningTime="2026-07-30 22:54:47.124869775 +0000 UTC m=+433.597333787" Jul 30 22:54:50.641623 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.641586 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x"] Jul 30 22:54:50.645521 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.645502 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:50.648175 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.648151 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tx2tw\"" Jul 30 22:54:50.658532 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.658510 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x"] Jul 30 22:54:50.732920 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.732887 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x64zq\" (UniqueName: \"kubernetes.io/projected/e052450e-f432-4201-8a20-fb2e7fab004d-kube-api-access-x64zq\") pod \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" (UID: \"e052450e-f432-4201-8a20-fb2e7fab004d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:50.733087 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.732952 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e052450e-f432-4201-8a20-fb2e7fab004d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" (UID: \"e052450e-f432-4201-8a20-fb2e7fab004d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:50.834296 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.834256 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x64zq\" (UniqueName: \"kubernetes.io/projected/e052450e-f432-4201-8a20-fb2e7fab004d-kube-api-access-x64zq\") pod \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" (UID: \"e052450e-f432-4201-8a20-fb2e7fab004d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:50.834493 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.834330 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e052450e-f432-4201-8a20-fb2e7fab004d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" (UID: \"e052450e-f432-4201-8a20-fb2e7fab004d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:50.834785 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.834766 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e052450e-f432-4201-8a20-fb2e7fab004d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" (UID: \"e052450e-f432-4201-8a20-fb2e7fab004d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:50.843897 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.843864 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x64zq\" (UniqueName: \"kubernetes.io/projected/e052450e-f432-4201-8a20-fb2e7fab004d-kube-api-access-x64zq\") pod \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" (UID: \"e052450e-f432-4201-8a20-fb2e7fab004d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:50.955886 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:50.955840 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:51.090148 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.090114 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x"] Jul 30 22:54:51.093187 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:54:51.093157 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode052450e_f432_4201_8a20_fb2e7fab004d.slice/crio-d9df75742fea00df0f1d82f433a16625cca0aa976a1d4aac81d2f288f47ef989 WatchSource:0}: Error finding container d9df75742fea00df0f1d82f433a16625cca0aa976a1d4aac81d2f288f47ef989: Status 404 returned error can't find the container with id d9df75742fea00df0f1d82f433a16625cca0aa976a1d4aac81d2f288f47ef989 Jul 30 22:54:51.122194 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.122164 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" event={"ID":"e052450e-f432-4201-8a20-fb2e7fab004d","Type":"ContainerStarted","Data":"d9df75742fea00df0f1d82f433a16625cca0aa976a1d4aac81d2f288f47ef989"} Jul 30 22:54:51.353414 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.353331 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x"] Jul 30 22:54:51.363407 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.363349 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x"] Jul 30 22:54:51.376244 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.376216 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6"] Jul 30 22:54:51.376422 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.376397 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" podUID="f812deb2-68f9-4c97-8211-340b1a6a8b44" containerName="manager" containerID="cri-o://a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550" gracePeriod=2 Jul 30 22:54:51.380333 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.380301 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj"] Jul 30 22:54:51.386435 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.386410 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:51.393436 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.393366 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6"] Jul 30 22:54:51.395788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.395627 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj"] Jul 30 22:54:51.400172 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:51.400141 2554 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f812deb2-68f9-4c97-8211-340b1a6a8b44\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-vzxk6\": pods \"limitador-operator-controller-manager-6c549b6d85-vzxk6\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:51.402248 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:51.402223 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-vzxk6\" is forbidden: User \"system:node:ip-10-0-131-130.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-130.ec2.internal' and this object" podUID="f812deb2-68f9-4c97-8211-340b1a6a8b44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:51.405084 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.405062 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w"] Jul 30 22:54:51.405383 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.405371 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f812deb2-68f9-4c97-8211-340b1a6a8b44" containerName="manager" Jul 30 22:54:51.405441 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.405384 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f812deb2-68f9-4c97-8211-340b1a6a8b44" containerName="manager" Jul 30 22:54:51.405500 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.405454 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f812deb2-68f9-4c97-8211-340b1a6a8b44" containerName="manager" Jul 30 22:54:51.408268 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.408252 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" Jul 30 22:54:51.431531 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.431510 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w"] Jul 30 22:54:51.439991 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.439969 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj5rg\" (UniqueName: \"kubernetes.io/projected/e4d21d8a-6942-48ce-8161-37b2c69110fa-kube-api-access-qj5rg\") pod \"kuadrant-operator-controller-manager-96db98c47-n5gbj\" (UID: \"e4d21d8a-6942-48ce-8161-37b2c69110fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:51.440105 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.440008 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4d21d8a-6942-48ce-8161-37b2c69110fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-n5gbj\" (UID: \"e4d21d8a-6942-48ce-8161-37b2c69110fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:51.454999 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:51.454972 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-vzxk6\" is forbidden: User \"system:node:ip-10-0-131-130.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-130.ec2.internal' and this object" podUID="f812deb2-68f9-4c97-8211-340b1a6a8b44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:51.541288 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.541245 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2x2p\" (UniqueName: \"kubernetes.io/projected/bfe06b27-7e9a-49ae-b271-abdd97d57d22-kube-api-access-v2x2p\") pod \"limitador-operator-controller-manager-6c549b6d85-mnk6w\" (UID: \"bfe06b27-7e9a-49ae-b271-abdd97d57d22\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" Jul 30 22:54:51.541456 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.541331 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj5rg\" (UniqueName: \"kubernetes.io/projected/e4d21d8a-6942-48ce-8161-37b2c69110fa-kube-api-access-qj5rg\") pod \"kuadrant-operator-controller-manager-96db98c47-n5gbj\" (UID: \"e4d21d8a-6942-48ce-8161-37b2c69110fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:51.541456 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.541360 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4d21d8a-6942-48ce-8161-37b2c69110fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-n5gbj\" (UID: \"e4d21d8a-6942-48ce-8161-37b2c69110fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:51.541797 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.541774 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4d21d8a-6942-48ce-8161-37b2c69110fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-n5gbj\" (UID: \"e4d21d8a-6942-48ce-8161-37b2c69110fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:51.559357 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.559324 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj5rg\" (UniqueName: \"kubernetes.io/projected/e4d21d8a-6942-48ce-8161-37b2c69110fa-kube-api-access-qj5rg\") pod \"kuadrant-operator-controller-manager-96db98c47-n5gbj\" (UID: \"e4d21d8a-6942-48ce-8161-37b2c69110fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:51.605850 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.605798 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:51.608063 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:51.608037 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-vzxk6\" is forbidden: User \"system:node:ip-10-0-131-130.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-130.ec2.internal' and this object" podUID="f812deb2-68f9-4c97-8211-340b1a6a8b44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:51.642579 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.642550 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2x2p\" (UniqueName: \"kubernetes.io/projected/bfe06b27-7e9a-49ae-b271-abdd97d57d22-kube-api-access-v2x2p\") pod \"limitador-operator-controller-manager-6c549b6d85-mnk6w\" (UID: \"bfe06b27-7e9a-49ae-b271-abdd97d57d22\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" Jul 30 22:54:51.653463 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.653438 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2x2p\" (UniqueName: \"kubernetes.io/projected/bfe06b27-7e9a-49ae-b271-abdd97d57d22-kube-api-access-v2x2p\") pod \"limitador-operator-controller-manager-6c549b6d85-mnk6w\" (UID: \"bfe06b27-7e9a-49ae-b271-abdd97d57d22\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" Jul 30 22:54:51.743033 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.743005 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xnvx\" (UniqueName: \"kubernetes.io/projected/f812deb2-68f9-4c97-8211-340b1a6a8b44-kube-api-access-8xnvx\") pod \"f812deb2-68f9-4c97-8211-340b1a6a8b44\" (UID: \"f812deb2-68f9-4c97-8211-340b1a6a8b44\") " Jul 30 22:54:51.745020 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.744992 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f812deb2-68f9-4c97-8211-340b1a6a8b44-kube-api-access-8xnvx" (OuterVolumeSpecName: "kube-api-access-8xnvx") pod "f812deb2-68f9-4c97-8211-340b1a6a8b44" (UID: "f812deb2-68f9-4c97-8211-340b1a6a8b44"). InnerVolumeSpecName "kube-api-access-8xnvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:54:51.765162 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.765139 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:51.771135 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.771116 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" Jul 30 22:54:51.844220 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.844123 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xnvx\" (UniqueName: \"kubernetes.io/projected/f812deb2-68f9-4c97-8211-340b1a6a8b44-kube-api-access-8xnvx\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:54:51.918552 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.918456 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj"] Jul 30 22:54:51.921885 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:54:51.921838 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d21d8a_6942_48ce_8161_37b2c69110fa.slice/crio-186097ae8381feb278b2bd0b0a7c9d18ff53234a9061a1db39ce35661323cd10 WatchSource:0}: Error finding container 186097ae8381feb278b2bd0b0a7c9d18ff53234a9061a1db39ce35661323cd10: Status 404 returned error can't find the container with id 186097ae8381feb278b2bd0b0a7c9d18ff53234a9061a1db39ce35661323cd10 Jul 30 22:54:51.942156 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:51.942131 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w"] Jul 30 22:54:51.944313 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:54:51.944280 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe06b27_7e9a_49ae_b271_abdd97d57d22.slice/crio-5f488573027e777442ae9775812bed88dc2a53167868ff825c8ea7b00628f7cd WatchSource:0}: Error finding container 5f488573027e777442ae9775812bed88dc2a53167868ff825c8ea7b00628f7cd: Status 404 returned error can't find the container with id 5f488573027e777442ae9775812bed88dc2a53167868ff825c8ea7b00628f7cd Jul 30 22:54:52.036869 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.036841 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f812deb2-68f9-4c97-8211-340b1a6a8b44" path="/var/lib/kubelet/pods/f812deb2-68f9-4c97-8211-340b1a6a8b44/volumes" Jul 30 22:54:52.130601 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.130512 2554 generic.go:356] "Generic (PLEG): container finished" podID="f812deb2-68f9-4c97-8211-340b1a6a8b44" containerID="a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550" exitCode=0 Jul 30 22:54:52.130601 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.130579 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vzxk6" Jul 30 22:54:52.130838 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.130621 2554 scope.go:117] "RemoveContainer" containerID="a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550" Jul 30 22:54:52.133337 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.133285 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" event={"ID":"bfe06b27-7e9a-49ae-b271-abdd97d57d22","Type":"ContainerStarted","Data":"769131e3e4a781bf435a3ec7ce86da3aed89d9fb93a00ce1367683b275ebc09a"} Jul 30 22:54:52.133453 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.133374 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" event={"ID":"bfe06b27-7e9a-49ae-b271-abdd97d57d22","Type":"ContainerStarted","Data":"5f488573027e777442ae9775812bed88dc2a53167868ff825c8ea7b00628f7cd"} Jul 30 22:54:52.135069 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.135043 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" event={"ID":"e4d21d8a-6942-48ce-8161-37b2c69110fa","Type":"ContainerStarted","Data":"186097ae8381feb278b2bd0b0a7c9d18ff53234a9061a1db39ce35661323cd10"} Jul 30 22:54:52.141273 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.141249 2554 scope.go:117] "RemoveContainer" containerID="a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550" Jul 30 22:54:52.141557 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:52.141530 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550\": container with ID starting with a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550 not found: ID does not exist" containerID="a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550" Jul 30 22:54:52.141622 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.141565 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550"} err="failed to get container status \"a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550\": rpc error: code = NotFound desc = could not find container \"a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550\": container with ID starting with a1788095ba00de40131138bd0369f6add1d369369201d6441be95703f3107550 not found: ID does not exist" Jul 30 22:54:52.151228 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:52.151188 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" podStartSLOduration=1.151174737 podStartE2EDuration="1.151174737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:54:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:54:52.149060151 +0000 UTC m=+438.621524165" watchObservedRunningTime="2026-07-30 22:54:52.151174737 +0000 UTC m=+438.623638749" Jul 30 22:54:53.143482 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:53.143447 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" Jul 30 22:54:56.167414 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.167366 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" event={"ID":"e4d21d8a-6942-48ce-8161-37b2c69110fa","Type":"ContainerStarted","Data":"a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167"} Jul 30 22:54:56.167848 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.167436 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:54:56.169053 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.169026 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" containerName="manager" containerID="cri-o://9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7" gracePeriod=2 Jul 30 22:54:56.187415 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:56.187378 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" is forbidden: User \"system:node:ip-10-0-131-130.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-130.ec2.internal' and this object" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:56.187974 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.187923 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" podStartSLOduration=1.298238032 podStartE2EDuration="5.18790622s" totalImagesPullingTime="3.889668188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:54:56.185286247 +0000 UTC m=+442.657750261" watchObservedRunningTime="2026-07-30 22:54:56.18790622 +0000 UTC m=+442.660370232" Jul 30 22:54:56.427028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.426975 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:56.429333 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:56.429304 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" is forbidden: User \"system:node:ip-10-0-131-130.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-130.ec2.internal' and this object" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:56.486776 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.486710 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x64zq\" (UniqueName: \"kubernetes.io/projected/e052450e-f432-4201-8a20-fb2e7fab004d-kube-api-access-x64zq\") pod \"e052450e-f432-4201-8a20-fb2e7fab004d\" (UID: \"e052450e-f432-4201-8a20-fb2e7fab004d\") " Jul 30 22:54:56.486988 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.486907 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e052450e-f432-4201-8a20-fb2e7fab004d-extensions-socket-volume\") pod \"e052450e-f432-4201-8a20-fb2e7fab004d\" (UID: \"e052450e-f432-4201-8a20-fb2e7fab004d\") " Jul 30 22:54:56.487220 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.487177 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e052450e-f432-4201-8a20-fb2e7fab004d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e052450e-f432-4201-8a20-fb2e7fab004d" (UID: "e052450e-f432-4201-8a20-fb2e7fab004d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 22:54:56.489083 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.489057 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e052450e-f432-4201-8a20-fb2e7fab004d-kube-api-access-x64zq" (OuterVolumeSpecName: "kube-api-access-x64zq") pod "e052450e-f432-4201-8a20-fb2e7fab004d" (UID: "e052450e-f432-4201-8a20-fb2e7fab004d"). InnerVolumeSpecName "kube-api-access-x64zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:54:56.588343 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.588312 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e052450e-f432-4201-8a20-fb2e7fab004d-extensions-socket-volume\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:54:56.588343 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:56.588341 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x64zq\" (UniqueName: \"kubernetes.io/projected/e052450e-f432-4201-8a20-fb2e7fab004d-kube-api-access-x64zq\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:54:57.176048 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:57.176014 2554 generic.go:356] "Generic (PLEG): container finished" podID="e052450e-f432-4201-8a20-fb2e7fab004d" containerID="9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7" exitCode=2 Jul 30 22:54:57.176422 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:57.176058 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:57.176422 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:57.176113 2554 scope.go:117] "RemoveContainer" containerID="9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7" Jul 30 22:54:57.178505 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:57.178482 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" is forbidden: User \"system:node:ip-10-0-131-130.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-130.ec2.internal' and this object" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:57.185070 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:57.185056 2554 scope.go:117] "RemoveContainer" containerID="9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7" Jul 30 22:54:57.185302 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:57.185283 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7\": container with ID starting with 9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7 not found: ID does not exist" containerID="9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7" Jul 30 22:54:57.185361 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:57.185314 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7"} err="failed to get container status \"9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7\": rpc error: code = NotFound desc = could not find container \"9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7\": container with ID starting with 9afc252a144774d3ea51d0bf57494dd43805fbcc9ab94e080be4df62b45344e7 not found: ID does not exist" Jul 30 22:54:57.186703 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:54:57.186678 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-q7q5x\" is forbidden: User \"system:node:ip-10-0-131-130.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-130.ec2.internal' and this object" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-q7q5x" Jul 30 22:54:58.036643 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:54:58.036610 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" path="/var/lib/kubelet/pods/e052450e-f432-4201-8a20-fb2e7fab004d/volumes" Jul 30 22:55:04.154242 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:04.154216 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-mnk6w" Jul 30 22:55:07.178720 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:07.178686 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:55:19.731191 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:19.731157 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj"] Jul 30 22:55:19.731654 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:19.731347 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" podUID="e4d21d8a-6942-48ce-8161-37b2c69110fa" containerName="manager" containerID="cri-o://a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167" gracePeriod=10 Jul 30 22:55:19.995567 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:19.995514 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:55:20.191052 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.191001 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4d21d8a-6942-48ce-8161-37b2c69110fa-extensions-socket-volume\") pod \"e4d21d8a-6942-48ce-8161-37b2c69110fa\" (UID: \"e4d21d8a-6942-48ce-8161-37b2c69110fa\") " Jul 30 22:55:20.191052 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.191056 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj5rg\" (UniqueName: \"kubernetes.io/projected/e4d21d8a-6942-48ce-8161-37b2c69110fa-kube-api-access-qj5rg\") pod \"e4d21d8a-6942-48ce-8161-37b2c69110fa\" (UID: \"e4d21d8a-6942-48ce-8161-37b2c69110fa\") " Jul 30 22:55:20.191534 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.191503 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d21d8a-6942-48ce-8161-37b2c69110fa-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e4d21d8a-6942-48ce-8161-37b2c69110fa" (UID: "e4d21d8a-6942-48ce-8161-37b2c69110fa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 22:55:20.193176 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.193155 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d21d8a-6942-48ce-8161-37b2c69110fa-kube-api-access-qj5rg" (OuterVolumeSpecName: "kube-api-access-qj5rg") pod "e4d21d8a-6942-48ce-8161-37b2c69110fa" (UID: "e4d21d8a-6942-48ce-8161-37b2c69110fa"). InnerVolumeSpecName "kube-api-access-qj5rg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:55:20.289222 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.289126 2554 generic.go:356] "Generic (PLEG): container finished" podID="e4d21d8a-6942-48ce-8161-37b2c69110fa" containerID="a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167" exitCode=0 Jul 30 22:55:20.289222 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.289176 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" Jul 30 22:55:20.289438 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.289217 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" event={"ID":"e4d21d8a-6942-48ce-8161-37b2c69110fa","Type":"ContainerDied","Data":"a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167"} Jul 30 22:55:20.289438 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.289271 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj" event={"ID":"e4d21d8a-6942-48ce-8161-37b2c69110fa","Type":"ContainerDied","Data":"186097ae8381feb278b2bd0b0a7c9d18ff53234a9061a1db39ce35661323cd10"} Jul 30 22:55:20.289438 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.289297 2554 scope.go:117] "RemoveContainer" containerID="a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167" Jul 30 22:55:20.291761 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.291719 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4d21d8a-6942-48ce-8161-37b2c69110fa-extensions-socket-volume\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:55:20.291883 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.291766 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj5rg\" (UniqueName: \"kubernetes.io/projected/e4d21d8a-6942-48ce-8161-37b2c69110fa-kube-api-access-qj5rg\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:55:20.298263 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.298247 2554 scope.go:117] "RemoveContainer" containerID="a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167" Jul 30 22:55:20.298528 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:55:20.298511 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167\": container with ID starting with a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167 not found: ID does not exist" containerID="a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167" Jul 30 22:55:20.298587 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.298535 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167"} err="failed to get container status \"a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167\": rpc error: code = NotFound desc = could not find container \"a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167\": container with ID starting with a3014e6cd3aef2683ae21293dc603e77c4a721e378d6a535879967c7f793c167 not found: ID does not exist" Jul 30 22:55:20.313313 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.313289 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj"] Jul 30 22:55:20.317588 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:20.317568 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-n5gbj"] Jul 30 22:55:22.035611 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:22.035576 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d21d8a-6942-48ce-8161-37b2c69110fa" path="/var/lib/kubelet/pods/e4d21d8a-6942-48ce-8161-37b2c69110fa/volumes" Jul 30 22:55:41.296876 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.296842 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zpcr2"] Jul 30 22:55:41.297410 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.297390 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d21d8a-6942-48ce-8161-37b2c69110fa" containerName="manager" Jul 30 22:55:41.297483 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.297413 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d21d8a-6942-48ce-8161-37b2c69110fa" containerName="manager" Jul 30 22:55:41.297483 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.297429 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" containerName="manager" Jul 30 22:55:41.297483 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.297437 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" containerName="manager" Jul 30 22:55:41.297627 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.297537 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d21d8a-6942-48ce-8161-37b2c69110fa" containerName="manager" Jul 30 22:55:41.297627 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.297551 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e052450e-f432-4201-8a20-fb2e7fab004d" containerName="manager" Jul 30 22:55:41.300487 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.300465 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:41.302875 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.302851 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 30 22:55:41.307289 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.307264 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zpcr2"] Jul 30 22:55:41.367184 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.367154 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-577jc\" (UniqueName: \"kubernetes.io/projected/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-kube-api-access-577jc\") pod \"limitador-limitador-859c4d96d6-zpcr2\" (UID: \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:41.367346 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.367209 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-config-file\") pod \"limitador-limitador-859c4d96d6-zpcr2\" (UID: \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:41.395036 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.395005 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zpcr2"] Jul 30 22:55:41.468526 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.468487 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-577jc\" (UniqueName: \"kubernetes.io/projected/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-kube-api-access-577jc\") pod \"limitador-limitador-859c4d96d6-zpcr2\" (UID: \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:41.468700 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.468549 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-config-file\") pod \"limitador-limitador-859c4d96d6-zpcr2\" (UID: \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:41.469152 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.469133 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-config-file\") pod \"limitador-limitador-859c4d96d6-zpcr2\" (UID: \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:41.476291 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.476269 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-577jc\" (UniqueName: \"kubernetes.io/projected/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-kube-api-access-577jc\") pod \"limitador-limitador-859c4d96d6-zpcr2\" (UID: \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:41.611418 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.611335 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:41.745495 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:41.745473 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zpcr2"] Jul 30 22:55:41.748629 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:55:41.748604 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca7fcaf_7084_4801_9af0_bdbe77e5b9b1.slice/crio-f6903e8c5a01d8636955e9bfae72c39c4513ca8905f162e27bde61414a114a9d WatchSource:0}: Error finding container f6903e8c5a01d8636955e9bfae72c39c4513ca8905f162e27bde61414a114a9d: Status 404 returned error can't find the container with id f6903e8c5a01d8636955e9bfae72c39c4513ca8905f162e27bde61414a114a9d Jul 30 22:55:42.396723 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:42.396687 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" event={"ID":"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1","Type":"ContainerStarted","Data":"f6903e8c5a01d8636955e9bfae72c39c4513ca8905f162e27bde61414a114a9d"} Jul 30 22:55:43.402723 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:43.402629 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" event={"ID":"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1","Type":"ContainerStarted","Data":"ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d"} Jul 30 22:55:43.403096 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:43.402761 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:43.418996 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:43.418947 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" podStartSLOduration=1.100755353 podStartE2EDuration="2.41893224s" totalImagesPullingTime="1.318176887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:55:43.416930459 +0000 UTC m=+489.889394472" watchObservedRunningTime="2026-07-30 22:55:43.41893224 +0000 UTC m=+489.891396277" Jul 30 22:55:54.408642 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:54.408610 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:55.898876 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:55.898840 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zpcr2"] Jul 30 22:55:55.899305 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:55.899077 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" podUID="aca7fcaf-7084-4801-9af0-bdbe77e5b9b1" containerName="limitador" containerID="cri-o://ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d" gracePeriod=30 Jul 30 22:55:56.450465 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.450435 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:56.461714 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.461652 2554 generic.go:356] "Generic (PLEG): container finished" podID="aca7fcaf-7084-4801-9af0-bdbe77e5b9b1" containerID="ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d" exitCode=0 Jul 30 22:55:56.461714 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.461691 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" Jul 30 22:55:56.461885 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.461761 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" event={"ID":"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1","Type":"ContainerDied","Data":"ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d"} Jul 30 22:55:56.461885 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.461814 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zpcr2" event={"ID":"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1","Type":"ContainerDied","Data":"f6903e8c5a01d8636955e9bfae72c39c4513ca8905f162e27bde61414a114a9d"} Jul 30 22:55:56.461885 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.461836 2554 scope.go:117] "RemoveContainer" containerID="ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d" Jul 30 22:55:56.471556 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.471534 2554 scope.go:117] "RemoveContainer" containerID="ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d" Jul 30 22:55:56.471860 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:55:56.471832 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d\": container with ID starting with ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d not found: ID does not exist" containerID="ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d" Jul 30 22:55:56.471952 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.471873 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d"} err="failed to get container status \"ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d\": rpc error: code = NotFound desc = could not find container \"ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d\": container with ID starting with ea5379914a873675cf1805fc96b806036dd8de290dd5d2d25ddb3f2ed312414d not found: ID does not exist" Jul 30 22:55:56.597866 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.597833 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-577jc\" (UniqueName: \"kubernetes.io/projected/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-kube-api-access-577jc\") pod \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\" (UID: \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\") " Jul 30 22:55:56.598071 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.597917 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-config-file\") pod \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\" (UID: \"aca7fcaf-7084-4801-9af0-bdbe77e5b9b1\") " Jul 30 22:55:56.598327 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.598303 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-config-file" (OuterVolumeSpecName: "config-file") pod "aca7fcaf-7084-4801-9af0-bdbe77e5b9b1" (UID: "aca7fcaf-7084-4801-9af0-bdbe77e5b9b1"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 22:55:56.599923 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.599895 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-kube-api-access-577jc" (OuterVolumeSpecName: "kube-api-access-577jc") pod "aca7fcaf-7084-4801-9af0-bdbe77e5b9b1" (UID: "aca7fcaf-7084-4801-9af0-bdbe77e5b9b1"). InnerVolumeSpecName "kube-api-access-577jc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:55:56.699094 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.699065 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-577jc\" (UniqueName: \"kubernetes.io/projected/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-kube-api-access-577jc\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:55:56.699094 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.699091 2554 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1-config-file\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:55:56.782511 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.782481 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zpcr2"] Jul 30 22:55:56.785961 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:56.785925 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zpcr2"] Jul 30 22:55:57.415468 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.415426 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6tncd"] Jul 30 22:55:57.415864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.415849 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aca7fcaf-7084-4801-9af0-bdbe77e5b9b1" containerName="limitador" Jul 30 22:55:57.415864 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.415865 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="aca7fcaf-7084-4801-9af0-bdbe77e5b9b1" containerName="limitador" Jul 30 22:55:57.415942 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.415932 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="aca7fcaf-7084-4801-9af0-bdbe77e5b9b1" containerName="limitador" Jul 30 22:55:57.418744 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.418718 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:55:57.421368 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.421337 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 22:55:57.421368 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.421338 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 30 22:55:57.422274 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.422254 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 30 22:55:57.422367 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.422260 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cpzbl\"" Jul 30 22:55:57.426809 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.426788 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6tncd"] Jul 30 22:55:57.507142 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.507110 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a7ea2c6d-d14c-40d7-99d7-99964f54211a-data\") pod \"postgres-69fb4594cf-6tncd\" (UID: \"a7ea2c6d-d14c-40d7-99d7-99964f54211a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:55:57.507303 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.507223 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6k6x\" (UniqueName: \"kubernetes.io/projected/a7ea2c6d-d14c-40d7-99d7-99964f54211a-kube-api-access-p6k6x\") pod \"postgres-69fb4594cf-6tncd\" (UID: \"a7ea2c6d-d14c-40d7-99d7-99964f54211a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:55:57.608014 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.607967 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6k6x\" (UniqueName: \"kubernetes.io/projected/a7ea2c6d-d14c-40d7-99d7-99964f54211a-kube-api-access-p6k6x\") pod \"postgres-69fb4594cf-6tncd\" (UID: \"a7ea2c6d-d14c-40d7-99d7-99964f54211a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:55:57.608014 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.608019 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a7ea2c6d-d14c-40d7-99d7-99964f54211a-data\") pod \"postgres-69fb4594cf-6tncd\" (UID: \"a7ea2c6d-d14c-40d7-99d7-99964f54211a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:55:57.608345 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.608329 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a7ea2c6d-d14c-40d7-99d7-99964f54211a-data\") pod \"postgres-69fb4594cf-6tncd\" (UID: \"a7ea2c6d-d14c-40d7-99d7-99964f54211a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:55:57.616594 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.616567 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6k6x\" (UniqueName: \"kubernetes.io/projected/a7ea2c6d-d14c-40d7-99d7-99964f54211a-kube-api-access-p6k6x\") pod \"postgres-69fb4594cf-6tncd\" (UID: \"a7ea2c6d-d14c-40d7-99d7-99964f54211a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:55:57.729620 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.729590 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:55:57.859961 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:57.859938 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6tncd"] Jul 30 22:55:57.862304 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:55:57.862271 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ea2c6d_d14c_40d7_99d7_99964f54211a.slice/crio-01474c88738d4faef86eaab7a6015176eaf84a09f172a7dd7126af04d409d8e1 WatchSource:0}: Error finding container 01474c88738d4faef86eaab7a6015176eaf84a09f172a7dd7126af04d409d8e1: Status 404 returned error can't find the container with id 01474c88738d4faef86eaab7a6015176eaf84a09f172a7dd7126af04d409d8e1 Jul 30 22:55:58.036047 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:58.035956 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aca7fcaf-7084-4801-9af0-bdbe77e5b9b1" path="/var/lib/kubelet/pods/aca7fcaf-7084-4801-9af0-bdbe77e5b9b1/volumes" Jul 30 22:55:58.474011 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:55:58.473979 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" event={"ID":"a7ea2c6d-d14c-40d7-99d7-99964f54211a","Type":"ContainerStarted","Data":"01474c88738d4faef86eaab7a6015176eaf84a09f172a7dd7126af04d409d8e1"} Jul 30 22:56:03.506700 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:03.506662 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" event={"ID":"a7ea2c6d-d14c-40d7-99d7-99964f54211a","Type":"ContainerStarted","Data":"1579f655885af05fd7579799091b4890f6b812a393fe1f9df9ba53c7979c19b3"} Jul 30 22:56:03.507080 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:03.506787 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:56:03.523588 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:03.523533 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" podStartSLOduration=1.6611036380000002 podStartE2EDuration="6.523513967s" totalImagesPullingTime="4.862410329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:56:03.520887049 +0000 UTC m=+509.993351061" watchObservedRunningTime="2026-07-30 22:56:03.523513967 +0000 UTC m=+509.995977980" Jul 30 22:56:09.539807 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:09.539779 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6tncd" Jul 30 22:56:16.613222 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.613187 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8f4b5dd99-qbgwf"] Jul 30 22:56:16.640105 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.640077 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8f4b5dd99-qbgwf"] Jul 30 22:56:16.640252 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.640166 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:16.642852 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.642826 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 30 22:56:16.642979 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.642897 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l4m8h\"" Jul 30 22:56:16.643955 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.643929 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 30 22:56:16.674796 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.674770 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/757936f9-8625-45ba-b9d0-bbdb72525444-webhook-cert\") pod \"maas-controller-8f4b5dd99-qbgwf\" (UID: \"757936f9-8625-45ba-b9d0-bbdb72525444\") " pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:16.674886 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.674833 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6zl5\" (UniqueName: \"kubernetes.io/projected/757936f9-8625-45ba-b9d0-bbdb72525444-kube-api-access-d6zl5\") pod \"maas-controller-8f4b5dd99-qbgwf\" (UID: \"757936f9-8625-45ba-b9d0-bbdb72525444\") " pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:16.775685 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.775640 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/757936f9-8625-45ba-b9d0-bbdb72525444-webhook-cert\") pod \"maas-controller-8f4b5dd99-qbgwf\" (UID: \"757936f9-8625-45ba-b9d0-bbdb72525444\") " pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:16.775900 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.775693 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6zl5\" (UniqueName: \"kubernetes.io/projected/757936f9-8625-45ba-b9d0-bbdb72525444-kube-api-access-d6zl5\") pod \"maas-controller-8f4b5dd99-qbgwf\" (UID: \"757936f9-8625-45ba-b9d0-bbdb72525444\") " pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:16.778789 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.778758 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/757936f9-8625-45ba-b9d0-bbdb72525444-webhook-cert\") pod \"maas-controller-8f4b5dd99-qbgwf\" (UID: \"757936f9-8625-45ba-b9d0-bbdb72525444\") " pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:16.779014 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.778995 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66ff69c46-qcp4v"] Jul 30 22:56:16.783757 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.783711 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6zl5\" (UniqueName: \"kubernetes.io/projected/757936f9-8625-45ba-b9d0-bbdb72525444-kube-api-access-d6zl5\") pod \"maas-controller-8f4b5dd99-qbgwf\" (UID: \"757936f9-8625-45ba-b9d0-bbdb72525444\") " pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:16.785836 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.785818 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:16.790897 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.790875 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66ff69c46-qcp4v"] Jul 30 22:56:16.877080 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.876992 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-webhook-cert\") pod \"maas-controller-66ff69c46-qcp4v\" (UID: \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\") " pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:16.877080 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.877029 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddncp\" (UniqueName: \"kubernetes.io/projected/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-kube-api-access-ddncp\") pod \"maas-controller-66ff69c46-qcp4v\" (UID: \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\") " pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:16.951614 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.951582 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:16.977585 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.977555 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-webhook-cert\") pod \"maas-controller-66ff69c46-qcp4v\" (UID: \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\") " pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:16.977711 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.977602 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddncp\" (UniqueName: \"kubernetes.io/projected/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-kube-api-access-ddncp\") pod \"maas-controller-66ff69c46-qcp4v\" (UID: \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\") " pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:16.979904 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.979884 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-webhook-cert\") pod \"maas-controller-66ff69c46-qcp4v\" (UID: \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\") " pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:16.985371 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:16.985350 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddncp\" (UniqueName: \"kubernetes.io/projected/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-kube-api-access-ddncp\") pod \"maas-controller-66ff69c46-qcp4v\" (UID: \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\") " pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:17.081936 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:17.081911 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8f4b5dd99-qbgwf"] Jul 30 22:56:17.084512 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:56:17.084481 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757936f9_8625_45ba_b9d0_bbdb72525444.slice/crio-7b8bf302cab1975550765ba8631c6bdfd2d64c91018b2316d0ccd69cbf5688b4 WatchSource:0}: Error finding container 7b8bf302cab1975550765ba8631c6bdfd2d64c91018b2316d0ccd69cbf5688b4: Status 404 returned error can't find the container with id 7b8bf302cab1975550765ba8631c6bdfd2d64c91018b2316d0ccd69cbf5688b4 Jul 30 22:56:17.105566 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:17.105539 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:17.234496 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:17.234472 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66ff69c46-qcp4v"] Jul 30 22:56:17.236309 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:56:17.236279 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb447e0a8_d42d_4a2e_a0d3_696511eb4df2.slice/crio-2e8018f8008ef99d57ab4d0846da0c30c06c4fed09df69462a1606a092e053a8 WatchSource:0}: Error finding container 2e8018f8008ef99d57ab4d0846da0c30c06c4fed09df69462a1606a092e053a8: Status 404 returned error can't find the container with id 2e8018f8008ef99d57ab4d0846da0c30c06c4fed09df69462a1606a092e053a8 Jul 30 22:56:17.570908 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:17.570872 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" event={"ID":"757936f9-8625-45ba-b9d0-bbdb72525444","Type":"ContainerStarted","Data":"7b8bf302cab1975550765ba8631c6bdfd2d64c91018b2316d0ccd69cbf5688b4"} Jul 30 22:56:17.571862 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:17.571839 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-qcp4v" event={"ID":"b447e0a8-d42d-4a2e-a0d3-696511eb4df2","Type":"ContainerStarted","Data":"2e8018f8008ef99d57ab4d0846da0c30c06c4fed09df69462a1606a092e053a8"} Jul 30 22:56:20.589801 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:20.589766 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" event={"ID":"757936f9-8625-45ba-b9d0-bbdb72525444","Type":"ContainerStarted","Data":"b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d"} Jul 30 22:56:20.590274 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:20.589859 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:20.591343 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:20.591322 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-qcp4v" event={"ID":"b447e0a8-d42d-4a2e-a0d3-696511eb4df2","Type":"ContainerStarted","Data":"ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1"} Jul 30 22:56:20.591451 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:20.591432 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:20.605913 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:20.605873 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" podStartSLOduration=2.120142246 podStartE2EDuration="4.605861888s" totalImagesPullingTime="2.485719642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:56:20.604159608 +0000 UTC m=+527.076623621" watchObservedRunningTime="2026-07-30 22:56:20.605861888 +0000 UTC m=+527.078325899" Jul 30 22:56:20.620787 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:20.620749 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66ff69c46-qcp4v" podStartSLOduration=2.29208676 podStartE2EDuration="4.620722916s" totalImagesPullingTime="2.328636156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:56:20.619070094 +0000 UTC m=+527.091534119" watchObservedRunningTime="2026-07-30 22:56:20.620722916 +0000 UTC m=+527.093186928" Jul 30 22:56:31.598228 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:31.598150 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:56:31.598641 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:31.598256 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:31.662416 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:31.662380 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8f4b5dd99-qbgwf"] Jul 30 22:56:31.662686 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:31.662628 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" podUID="757936f9-8625-45ba-b9d0-bbdb72525444" containerName="manager" containerID="cri-o://b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d" gracePeriod=30 Jul 30 22:56:31.908599 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:31.908572 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:32.006300 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.006265 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6zl5\" (UniqueName: \"kubernetes.io/projected/757936f9-8625-45ba-b9d0-bbdb72525444-kube-api-access-d6zl5\") pod \"757936f9-8625-45ba-b9d0-bbdb72525444\" (UID: \"757936f9-8625-45ba-b9d0-bbdb72525444\") " Jul 30 22:56:32.006501 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.006326 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/757936f9-8625-45ba-b9d0-bbdb72525444-webhook-cert\") pod \"757936f9-8625-45ba-b9d0-bbdb72525444\" (UID: \"757936f9-8625-45ba-b9d0-bbdb72525444\") " Jul 30 22:56:32.008587 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.008554 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/757936f9-8625-45ba-b9d0-bbdb72525444-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "757936f9-8625-45ba-b9d0-bbdb72525444" (UID: "757936f9-8625-45ba-b9d0-bbdb72525444"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 22:56:32.008698 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.008614 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757936f9-8625-45ba-b9d0-bbdb72525444-kube-api-access-d6zl5" (OuterVolumeSpecName: "kube-api-access-d6zl5") pod "757936f9-8625-45ba-b9d0-bbdb72525444" (UID: "757936f9-8625-45ba-b9d0-bbdb72525444"). InnerVolumeSpecName "kube-api-access-d6zl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:56:32.107788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.107755 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6zl5\" (UniqueName: \"kubernetes.io/projected/757936f9-8625-45ba-b9d0-bbdb72525444-kube-api-access-d6zl5\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:56:32.107788 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.107783 2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/757936f9-8625-45ba-b9d0-bbdb72525444-webhook-cert\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:56:32.649072 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.649040 2554 generic.go:356] "Generic (PLEG): container finished" podID="757936f9-8625-45ba-b9d0-bbdb72525444" containerID="b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d" exitCode=0 Jul 30 22:56:32.649487 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.649082 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" Jul 30 22:56:32.649487 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.649090 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" event={"ID":"757936f9-8625-45ba-b9d0-bbdb72525444","Type":"ContainerDied","Data":"b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d"} Jul 30 22:56:32.649487 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.649136 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8f4b5dd99-qbgwf" event={"ID":"757936f9-8625-45ba-b9d0-bbdb72525444","Type":"ContainerDied","Data":"7b8bf302cab1975550765ba8631c6bdfd2d64c91018b2316d0ccd69cbf5688b4"} Jul 30 22:56:32.649487 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.649157 2554 scope.go:117] "RemoveContainer" containerID="b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d" Jul 30 22:56:32.657925 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.657906 2554 scope.go:117] "RemoveContainer" containerID="b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d" Jul 30 22:56:32.658194 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:56:32.658173 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d\": container with ID starting with b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d not found: ID does not exist" containerID="b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d" Jul 30 22:56:32.658255 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.658206 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d"} err="failed to get container status \"b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d\": rpc error: code = NotFound desc = could not find container \"b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d\": container with ID starting with b2f0164dc5bd17f280f2eb3ce2f78b78130037c496c92a46e1344a851576103d not found: ID does not exist" Jul 30 22:56:32.665293 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.665269 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8f4b5dd99-qbgwf"] Jul 30 22:56:32.668601 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:32.668582 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8f4b5dd99-qbgwf"] Jul 30 22:56:34.042671 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:34.042635 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="757936f9-8625-45ba-b9d0-bbdb72525444" path="/var/lib/kubelet/pods/757936f9-8625-45ba-b9d0-bbdb72525444/volumes" Jul 30 22:56:53.390551 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.390520 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8"] Jul 30 22:56:53.391141 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.391121 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757936f9-8625-45ba-b9d0-bbdb72525444" containerName="manager" Jul 30 22:56:53.391229 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.391144 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="757936f9-8625-45ba-b9d0-bbdb72525444" containerName="manager" Jul 30 22:56:53.391282 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.391236 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="757936f9-8625-45ba-b9d0-bbdb72525444" containerName="manager" Jul 30 22:56:53.399350 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.399329 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.402982 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.402891 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 30 22:56:53.403653 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.402895 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rqx9x\"" Jul 30 22:56:53.405106 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.405063 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8"] Jul 30 22:56:53.483232 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.483187 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-maas-api-tls\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.483496 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.483474 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-openshift-service-ca\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.483602 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.483534 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll8v5\" (UniqueName: \"kubernetes.io/projected/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-kube-api-access-ll8v5\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.584833 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.584803 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-openshift-service-ca\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.585009 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.584855 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll8v5\" (UniqueName: \"kubernetes.io/projected/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-kube-api-access-ll8v5\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.585079 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.585002 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-maas-api-tls\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.585519 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.585493 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-openshift-service-ca\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.587359 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.587340 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-maas-api-tls\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.595249 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.595229 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll8v5\" (UniqueName: \"kubernetes.io/projected/fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3-kube-api-access-ll8v5\") pod \"maas-api-7dcf74846d-h78k8\" (UID: \"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3\") " pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.710998 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.710958 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:53.847884 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.847648 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8"] Jul 30 22:56:53.849364 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.849340 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s"] Jul 30 22:56:53.850301 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:56:53.850274 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa073fc7_6d1d_4da7_b0c9_0ae5c6272fb3.slice/crio-41492f56c634b4e1dc5dc015513f5219ef9a2dbd0f1cf6bf55a9af4d65b6f7d7 WatchSource:0}: Error finding container 41492f56c634b4e1dc5dc015513f5219ef9a2dbd0f1cf6bf55a9af4d65b6f7d7: Status 404 returned error can't find the container with id 41492f56c634b4e1dc5dc015513f5219ef9a2dbd0f1cf6bf55a9af4d65b6f7d7 Jul 30 22:56:53.855353 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.855334 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:53.857772 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.857746 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 30 22:56:53.857860 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.857726 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lhf27\"" Jul 30 22:56:53.863183 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.863121 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s"] Jul 30 22:56:53.977211 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.977136 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cq57g"] Jul 30 22:56:53.981455 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.981434 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:53.988644 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.988566 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d7648de-0da2-43be-abf6-d74241ca1f9b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-h9b6s\" (UID: \"0d7648de-0da2-43be-abf6-d74241ca1f9b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:53.989263 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.989133 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62fjs\" (UniqueName: \"kubernetes.io/projected/0d7648de-0da2-43be-abf6-d74241ca1f9b-kube-api-access-62fjs\") pod \"payload-pre-processing-687dbb6cf6-h9b6s\" (UID: \"0d7648de-0da2-43be-abf6-d74241ca1f9b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:53.991861 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:53.991836 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cq57g"] Jul 30 22:56:54.090062 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.090018 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpwsd\" (UniqueName: \"kubernetes.io/projected/4149d8bc-83ab-4403-bb3b-b43ba23413a0-kube-api-access-jpwsd\") pod \"payload-processing-5cc6cd444b-cq57g\" (UID: \"4149d8bc-83ab-4403-bb3b-b43ba23413a0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:54.090297 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.090082 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d7648de-0da2-43be-abf6-d74241ca1f9b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-h9b6s\" (UID: \"0d7648de-0da2-43be-abf6-d74241ca1f9b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:54.090297 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.090157 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62fjs\" (UniqueName: \"kubernetes.io/projected/0d7648de-0da2-43be-abf6-d74241ca1f9b-kube-api-access-62fjs\") pod \"payload-pre-processing-687dbb6cf6-h9b6s\" (UID: \"0d7648de-0da2-43be-abf6-d74241ca1f9b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:54.090297 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.090201 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4149d8bc-83ab-4403-bb3b-b43ba23413a0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cq57g\" (UID: \"4149d8bc-83ab-4403-bb3b-b43ba23413a0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:54.090934 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.090912 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d7648de-0da2-43be-abf6-d74241ca1f9b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-h9b6s\" (UID: \"0d7648de-0da2-43be-abf6-d74241ca1f9b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:54.098033 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.098009 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62fjs\" (UniqueName: \"kubernetes.io/projected/0d7648de-0da2-43be-abf6-d74241ca1f9b-kube-api-access-62fjs\") pod \"payload-pre-processing-687dbb6cf6-h9b6s\" (UID: \"0d7648de-0da2-43be-abf6-d74241ca1f9b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:54.169859 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.169821 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:54.190876 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.190849 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpwsd\" (UniqueName: \"kubernetes.io/projected/4149d8bc-83ab-4403-bb3b-b43ba23413a0-kube-api-access-jpwsd\") pod \"payload-processing-5cc6cd444b-cq57g\" (UID: \"4149d8bc-83ab-4403-bb3b-b43ba23413a0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:54.190998 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.190931 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4149d8bc-83ab-4403-bb3b-b43ba23413a0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cq57g\" (UID: \"4149d8bc-83ab-4403-bb3b-b43ba23413a0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:54.191629 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.191604 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4149d8bc-83ab-4403-bb3b-b43ba23413a0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cq57g\" (UID: \"4149d8bc-83ab-4403-bb3b-b43ba23413a0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:54.198812 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.198789 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpwsd\" (UniqueName: \"kubernetes.io/projected/4149d8bc-83ab-4403-bb3b-b43ba23413a0-kube-api-access-jpwsd\") pod \"payload-processing-5cc6cd444b-cq57g\" (UID: \"4149d8bc-83ab-4403-bb3b-b43ba23413a0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:54.292554 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.292529 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:54.309583 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.309561 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s"] Jul 30 22:56:54.311685 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:56:54.311651 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d7648de_0da2_43be_abf6_d74241ca1f9b.slice/crio-b927d75e354465af71d71b4b89d27469230bbec1e63422f34c922d43f5eb1e41 WatchSource:0}: Error finding container b927d75e354465af71d71b4b89d27469230bbec1e63422f34c922d43f5eb1e41: Status 404 returned error can't find the container with id b927d75e354465af71d71b4b89d27469230bbec1e63422f34c922d43f5eb1e41 Jul 30 22:56:54.431756 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.431714 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cq57g"] Jul 30 22:56:54.433805 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:56:54.433777 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4149d8bc_83ab_4403_bb3b_b43ba23413a0.slice/crio-633603c8e2e6a319826e22830ab5c77a3753fafe80f046404f84ddde59cf52d2 WatchSource:0}: Error finding container 633603c8e2e6a319826e22830ab5c77a3753fafe80f046404f84ddde59cf52d2: Status 404 returned error can't find the container with id 633603c8e2e6a319826e22830ab5c77a3753fafe80f046404f84ddde59cf52d2 Jul 30 22:56:54.756373 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.756328 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" event={"ID":"4149d8bc-83ab-4403-bb3b-b43ba23413a0","Type":"ContainerStarted","Data":"633603c8e2e6a319826e22830ab5c77a3753fafe80f046404f84ddde59cf52d2"} Jul 30 22:56:54.757715 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.757685 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" event={"ID":"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3","Type":"ContainerStarted","Data":"41492f56c634b4e1dc5dc015513f5219ef9a2dbd0f1cf6bf55a9af4d65b6f7d7"} Jul 30 22:56:54.759193 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:54.759152 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" event={"ID":"0d7648de-0da2-43be-abf6-d74241ca1f9b","Type":"ContainerStarted","Data":"b927d75e354465af71d71b4b89d27469230bbec1e63422f34c922d43f5eb1e41"} Jul 30 22:56:55.769061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:55.768969 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" event={"ID":"fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3","Type":"ContainerStarted","Data":"191a5200b5682261ce4f75a3aebace5e2991ca48059d921f0d4862bb1ded38a6"} Jul 30 22:56:55.769490 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:55.769133 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:56:55.784777 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:55.784639 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" podStartSLOduration=1.127796122 podStartE2EDuration="2.784618063s" totalImagesPullingTime="1.656821941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:56:55.784247212 +0000 UTC m=+562.256711248" watchObservedRunningTime="2026-07-30 22:56:55.784618063 +0000 UTC m=+562.257082076" Jul 30 22:56:57.781632 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:57.781597 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" event={"ID":"4149d8bc-83ab-4403-bb3b-b43ba23413a0","Type":"ContainerStarted","Data":"25b34d4c41d164fad53d0b741a9b6719522b276bae152dff7a429c4f6dccd1ad"} Jul 30 22:56:57.782187 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:57.781684 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:56:57.783206 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:57.783180 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" event={"ID":"0d7648de-0da2-43be-abf6-d74241ca1f9b","Type":"ContainerStarted","Data":"62e3fbc08de82703550113b1a21b93d428863ca107f55e27a9ece72af4ca4f9e"} Jul 30 22:56:57.783317 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:57.783220 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:56:57.797218 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:57.797174 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" podStartSLOduration=2.213053763 podStartE2EDuration="4.797159337s" totalImagesPullingTime="2.584105574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:56:57.796744675 +0000 UTC m=+564.269208684" watchObservedRunningTime="2026-07-30 22:56:57.797159337 +0000 UTC m=+564.269623349" Jul 30 22:56:57.811102 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:56:57.811064 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" podStartSLOduration=2.113923556 podStartE2EDuration="4.811049021s" totalImagesPullingTime="2.697125465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:56:57.810274239 +0000 UTC m=+564.282738263" watchObservedRunningTime="2026-07-30 22:56:57.811049021 +0000 UTC m=+564.283513034" Jul 30 22:57:01.781609 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:01.781582 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7dcf74846d-h78k8" Jul 30 22:57:08.789699 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:08.789664 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-h9b6s" Jul 30 22:57:08.790169 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:08.789848 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-cq57g" Jul 30 22:57:10.844750 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.844704 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv"] Jul 30 22:57:10.849485 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.849461 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:10.853028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.853000 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 30 22:57:10.853188 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.853029 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mkjq5\"" Jul 30 22:57:10.853516 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.853495 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 30 22:57:10.853622 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.853582 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 30 22:57:10.856111 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.856087 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv"] Jul 30 22:57:10.935040 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.935006 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a067d259-83ae-42b2-ab12-caf6124c104c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:10.935235 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.935064 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4pff\" (UniqueName: \"kubernetes.io/projected/a067d259-83ae-42b2-ab12-caf6124c104c-kube-api-access-v4pff\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:10.935235 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.935090 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:10.935235 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.935113 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:10.935235 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.935199 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:10.935447 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:10.935239 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.035795 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.035762 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4pff\" (UniqueName: \"kubernetes.io/projected/a067d259-83ae-42b2-ab12-caf6124c104c-kube-api-access-v4pff\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.035795 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.035799 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.036028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.035818 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.036028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.035882 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.036028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.035931 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.036028 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.035955 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a067d259-83ae-42b2-ab12-caf6124c104c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.036273 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.036251 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.036344 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.036318 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.036398 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.036352 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.038224 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.038205 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a067d259-83ae-42b2-ab12-caf6124c104c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.038492 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.038476 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a067d259-83ae-42b2-ab12-caf6124c104c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.042940 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.042918 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4pff\" (UniqueName: \"kubernetes.io/projected/a067d259-83ae-42b2-ab12-caf6124c104c-kube-api-access-v4pff\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv\" (UID: \"a067d259-83ae-42b2-ab12-caf6124c104c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.161412 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.161320 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:11.301070 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.301002 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv"] Jul 30 22:57:11.303505 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:57:11.303473 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda067d259_83ae_42b2_ab12_caf6124c104c.slice/crio-81b4e93ba790acf93f84e8955201fdf62c203e1136362170ca725ce58d33d8a9 WatchSource:0}: Error finding container 81b4e93ba790acf93f84e8955201fdf62c203e1136362170ca725ce58d33d8a9: Status 404 returned error can't find the container with id 81b4e93ba790acf93f84e8955201fdf62c203e1136362170ca725ce58d33d8a9 Jul 30 22:57:11.855995 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:11.855951 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" event={"ID":"a067d259-83ae-42b2-ab12-caf6124c104c","Type":"ContainerStarted","Data":"81b4e93ba790acf93f84e8955201fdf62c203e1136362170ca725ce58d33d8a9"} Jul 30 22:57:12.862332 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:12.862245 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" event={"ID":"a067d259-83ae-42b2-ab12-caf6124c104c","Type":"ContainerStarted","Data":"3699d4a5587741a7da21db90e1bf06cd4692929c1bf732232b0a64c44fca8e2e"} Jul 30 22:57:12.878920 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:12.878870 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" podStartSLOduration=1.753261144 podStartE2EDuration="2.878855433s" totalImagesPullingTime="1.125594289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:57:12.876616523 +0000 UTC m=+579.349080560" watchObservedRunningTime="2026-07-30 22:57:12.878855433 +0000 UTC m=+579.351319445" Jul 30 22:57:15.196384 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.196353 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc"] Jul 30 22:57:15.200063 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.200029 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.202937 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.202915 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 30 22:57:15.223382 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.223354 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc"] Jul 30 22:57:15.374940 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.374905 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/143b975a-e272-4624-badf-de435e717efb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.374940 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.374949 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.375181 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.374971 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.375181 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.375060 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8szps\" (UniqueName: \"kubernetes.io/projected/143b975a-e272-4624-badf-de435e717efb-kube-api-access-8szps\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.375181 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.375121 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.375181 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.375139 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.476318 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476230 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.476318 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476263 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.476547 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476356 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/143b975a-e272-4624-badf-de435e717efb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.476547 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476390 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.476547 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476423 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.476547 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476479 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8szps\" (UniqueName: \"kubernetes.io/projected/143b975a-e272-4624-badf-de435e717efb-kube-api-access-8szps\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.476861 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476826 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.476861 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476854 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.477033 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.476910 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.478644 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.478615 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/143b975a-e272-4624-badf-de435e717efb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.478945 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.478927 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/143b975a-e272-4624-badf-de435e717efb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.483470 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.483450 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8szps\" (UniqueName: \"kubernetes.io/projected/143b975a-e272-4624-badf-de435e717efb-kube-api-access-8szps\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6ncc\" (UID: \"143b975a-e272-4624-badf-de435e717efb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.511532 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.511500 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:15.652071 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.652047 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc"] Jul 30 22:57:15.653964 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:57:15.653926 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143b975a_e272_4624_badf_de435e717efb.slice/crio-fe205ced68c8daac341297f00b9663c79854c175f2f96572829c3f4428658d85 WatchSource:0}: Error finding container fe205ced68c8daac341297f00b9663c79854c175f2f96572829c3f4428658d85: Status 404 returned error can't find the container with id fe205ced68c8daac341297f00b9663c79854c175f2f96572829c3f4428658d85 Jul 30 22:57:15.877670 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:15.877586 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" event={"ID":"143b975a-e272-4624-badf-de435e717efb","Type":"ContainerStarted","Data":"fe205ced68c8daac341297f00b9663c79854c175f2f96572829c3f4428658d85"} Jul 30 22:57:16.884224 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:16.884183 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" event={"ID":"143b975a-e272-4624-badf-de435e717efb","Type":"ContainerStarted","Data":"9b2105cb269e2d33b4e646da0347199cbd8b648d70e09ed01f1c2fd2a2394b50"} Jul 30 22:57:16.900881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:16.900836 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" podStartSLOduration=1.661674717 podStartE2EDuration="1.900821871s" totalImagesPullingTime="239.147154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:57:16.899533159 +0000 UTC m=+583.371997172" watchObservedRunningTime="2026-07-30 22:57:16.900821871 +0000 UTC m=+583.373285883" Jul 30 22:57:21.161802 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:21.161767 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:21.162216 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:21.161821 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:21.174174 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:21.174149 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:21.918609 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:21.918579 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv" Jul 30 22:57:25.512533 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:25.512502 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:25.512972 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:25.512806 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:25.525189 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:25.525169 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:25.937383 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:25.937353 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6ncc" Jul 30 22:57:33.973349 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:33.973321 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 22:57:33.975609 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:33.975589 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 22:57:53.142645 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.142608 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks"] Jul 30 22:57:53.146897 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.146875 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.149911 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.149888 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 30 22:57:53.157276 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.157249 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks"] Jul 30 22:57:53.186103 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.186062 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.186264 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.186148 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.186264 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.186214 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.186264 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.186246 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.186401 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.186264 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4acb68-8444-41d8-8805-8651a7cc550a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.186401 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.186376 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r95qx\" (UniqueName: \"kubernetes.io/projected/5c4acb68-8444-41d8-8805-8651a7cc550a-kube-api-access-r95qx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287302 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287263 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r95qx\" (UniqueName: \"kubernetes.io/projected/5c4acb68-8444-41d8-8805-8651a7cc550a-kube-api-access-r95qx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287302 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287302 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287528 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287440 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287528 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287499 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287634 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287538 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287634 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287565 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4acb68-8444-41d8-8805-8651a7cc550a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287880 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287857 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287962 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287880 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.287962 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.287952 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.289595 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.289577 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4acb68-8444-41d8-8805-8651a7cc550a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.290147 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.290127 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4acb68-8444-41d8-8805-8651a7cc550a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.295527 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.295506 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r95qx\" (UniqueName: \"kubernetes.io/projected/5c4acb68-8444-41d8-8805-8651a7cc550a-kube-api-access-r95qx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhhks\" (UID: \"5c4acb68-8444-41d8-8805-8651a7cc550a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.458710 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.458665 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:57:53.595054 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.595008 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks"] Jul 30 22:57:53.597552 ip-10-0-131-130 kubenswrapper[2554]: W0730 22:57:53.597524 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c4acb68_8444_41d8_8805_8651a7cc550a.slice/crio-84f9c7b3ee82b5d2a53e94e6ff8f8844fde892dc06fe2690d1c33f35818ee657 WatchSource:0}: Error finding container 84f9c7b3ee82b5d2a53e94e6ff8f8844fde892dc06fe2690d1c33f35818ee657: Status 404 returned error can't find the container with id 84f9c7b3ee82b5d2a53e94e6ff8f8844fde892dc06fe2690d1c33f35818ee657 Jul 30 22:57:53.600451 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:53.599941 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 22:57:54.074039 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:54.073949 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" event={"ID":"5c4acb68-8444-41d8-8805-8651a7cc550a","Type":"ContainerStarted","Data":"dc82d94b7b9325e148168ce9d98e95472531a29de2ae04395090c4e94feb00c8"} Jul 30 22:57:54.074039 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:54.074011 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" event={"ID":"5c4acb68-8444-41d8-8805-8651a7cc550a","Type":"ContainerStarted","Data":"84f9c7b3ee82b5d2a53e94e6ff8f8844fde892dc06fe2690d1c33f35818ee657"} Jul 30 22:57:54.091457 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:57:54.091406 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" podStartSLOduration=0.916707159 podStartE2EDuration="1.091391171s" totalImagesPullingTime="174.684012ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:57:54.089443002 +0000 UTC m=+620.561907014" watchObservedRunningTime="2026-07-30 22:57:54.091391171 +0000 UTC m=+620.563855182" Jul 30 22:58:03.459706 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:58:03.459664 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:58:03.460246 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:58:03.459725 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:58:03.473190 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:58:03.473161 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:58:04.141564 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:58:04.141537 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhhks" Jul 30 22:59:55.402302 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.402266 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66ff69c46-qcp4v"] Jul 30 22:59:55.402758 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.402484 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66ff69c46-qcp4v" podUID="b447e0a8-d42d-4a2e-a0d3-696511eb4df2" containerName="manager" containerID="cri-o://ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1" gracePeriod=30 Jul 30 22:59:55.656703 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.656641 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:59:55.701646 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.701606 2554 generic.go:356] "Generic (PLEG): container finished" podID="b447e0a8-d42d-4a2e-a0d3-696511eb4df2" containerID="ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1" exitCode=0 Jul 30 22:59:55.701868 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.701656 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-qcp4v" Jul 30 22:59:55.701868 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.701684 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-qcp4v" event={"ID":"b447e0a8-d42d-4a2e-a0d3-696511eb4df2","Type":"ContainerDied","Data":"ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1"} Jul 30 22:59:55.701868 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.701758 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-qcp4v" event={"ID":"b447e0a8-d42d-4a2e-a0d3-696511eb4df2","Type":"ContainerDied","Data":"2e8018f8008ef99d57ab4d0846da0c30c06c4fed09df69462a1606a092e053a8"} Jul 30 22:59:55.701868 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.701798 2554 scope.go:117] "RemoveContainer" containerID="ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1" Jul 30 22:59:55.710881 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.710857 2554 scope.go:117] "RemoveContainer" containerID="ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1" Jul 30 22:59:55.711189 ip-10-0-131-130 kubenswrapper[2554]: E0730 22:59:55.711166 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1\": container with ID starting with ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1 not found: ID does not exist" containerID="ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1" Jul 30 22:59:55.711255 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.711202 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1"} err="failed to get container status \"ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1\": rpc error: code = NotFound desc = could not find container \"ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1\": container with ID starting with ed89b1e15317f7d0cf12655bb44c30e5c8131fec8de45e3c2e8e36a11d7b8ed1 not found: ID does not exist" Jul 30 22:59:55.771895 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.771856 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddncp\" (UniqueName: \"kubernetes.io/projected/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-kube-api-access-ddncp\") pod \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\" (UID: \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\") " Jul 30 22:59:55.772061 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.771945 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-webhook-cert\") pod \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\" (UID: \"b447e0a8-d42d-4a2e-a0d3-696511eb4df2\") " Jul 30 22:59:55.774043 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.774009 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-kube-api-access-ddncp" (OuterVolumeSpecName: "kube-api-access-ddncp") pod "b447e0a8-d42d-4a2e-a0d3-696511eb4df2" (UID: "b447e0a8-d42d-4a2e-a0d3-696511eb4df2"). InnerVolumeSpecName "kube-api-access-ddncp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 22:59:55.774181 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.774048 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b447e0a8-d42d-4a2e-a0d3-696511eb4df2" (UID: "b447e0a8-d42d-4a2e-a0d3-696511eb4df2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 22:59:55.872834 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.872775 2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-webhook-cert\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:59:55.872834 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:55.872808 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddncp\" (UniqueName: \"kubernetes.io/projected/b447e0a8-d42d-4a2e-a0d3-696511eb4df2-kube-api-access-ddncp\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 22:59:56.023396 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:56.023362 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66ff69c46-qcp4v"] Jul 30 22:59:56.026682 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:56.026655 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66ff69c46-qcp4v"] Jul 30 22:59:56.037275 ip-10-0-131-130 kubenswrapper[2554]: I0730 22:59:56.037244 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b447e0a8-d42d-4a2e-a0d3-696511eb4df2" path="/var/lib/kubelet/pods/b447e0a8-d42d-4a2e-a0d3-696511eb4df2/volumes" Jul 30 23:00:56.817253 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.817179 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66ff69c46-4gzjf"] Jul 30 23:00:56.817682 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.817591 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b447e0a8-d42d-4a2e-a0d3-696511eb4df2" containerName="manager" Jul 30 23:00:56.817682 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.817602 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b447e0a8-d42d-4a2e-a0d3-696511eb4df2" containerName="manager" Jul 30 23:00:56.817682 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.817662 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b447e0a8-d42d-4a2e-a0d3-696511eb4df2" containerName="manager" Jul 30 23:00:56.822107 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.822084 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:56.825031 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.825008 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 30 23:00:56.826128 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.826103 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l4m8h\"" Jul 30 23:00:56.826234 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.826114 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 30 23:00:56.831993 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.831968 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66ff69c46-4gzjf"] Jul 30 23:00:56.912887 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.912846 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsmnn\" (UniqueName: \"kubernetes.io/projected/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-kube-api-access-rsmnn\") pod \"maas-controller-66ff69c46-4gzjf\" (UID: \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\") " pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:56.913071 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:56.912919 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-webhook-cert\") pod \"maas-controller-66ff69c46-4gzjf\" (UID: \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\") " pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:57.013876 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:57.013841 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsmnn\" (UniqueName: \"kubernetes.io/projected/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-kube-api-access-rsmnn\") pod \"maas-controller-66ff69c46-4gzjf\" (UID: \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\") " pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:57.014068 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:57.013924 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-webhook-cert\") pod \"maas-controller-66ff69c46-4gzjf\" (UID: \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\") " pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:57.016315 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:57.016287 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-webhook-cert\") pod \"maas-controller-66ff69c46-4gzjf\" (UID: \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\") " pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:57.024404 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:57.024383 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsmnn\" (UniqueName: \"kubernetes.io/projected/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-kube-api-access-rsmnn\") pod \"maas-controller-66ff69c46-4gzjf\" (UID: \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\") " pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:57.134117 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:57.134025 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:57.279362 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:57.279339 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66ff69c46-4gzjf"] Jul 30 23:00:57.281461 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:00:57.281422 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f3cd3b_5c00_4aa6_9554_ef305a23e7dd.slice/crio-57bcb147325168eb39d3368b1a7ea970d9d80a7fd557419b704f5d2489dfe0ea WatchSource:0}: Error finding container 57bcb147325168eb39d3368b1a7ea970d9d80a7fd557419b704f5d2489dfe0ea: Status 404 returned error can't find the container with id 57bcb147325168eb39d3368b1a7ea970d9d80a7fd557419b704f5d2489dfe0ea Jul 30 23:00:58.010940 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:58.010855 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-4gzjf" event={"ID":"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd","Type":"ContainerStarted","Data":"b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3"} Jul 30 23:00:58.010940 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:58.010910 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-4gzjf" event={"ID":"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd","Type":"ContainerStarted","Data":"57bcb147325168eb39d3368b1a7ea970d9d80a7fd557419b704f5d2489dfe0ea"} Jul 30 23:00:58.011319 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:58.011029 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:00:58.031702 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:00:58.031654 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66ff69c46-4gzjf" podStartSLOduration=1.607020449 podStartE2EDuration="2.031639954s" totalImagesPullingTime="424.619505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:00:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:00:58.029755759 +0000 UTC m=+804.502219772" watchObservedRunningTime="2026-07-30 23:00:58.031639954 +0000 UTC m=+804.504103966" Jul 30 23:01:09.019417 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:09.019385 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:01:13.716346 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:13.716310 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66ff69c46-4gzjf"] Jul 30 23:01:13.716823 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:13.716504 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66ff69c46-4gzjf" podUID="e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd" containerName="manager" containerID="cri-o://b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3" gracePeriod=30 Jul 30 23:01:13.970064 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:13.970004 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:01:14.075483 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.075447 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsmnn\" (UniqueName: \"kubernetes.io/projected/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-kube-api-access-rsmnn\") pod \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\" (UID: \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\") " Jul 30 23:01:14.075675 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.075518 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-webhook-cert\") pod \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\" (UID: \"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd\") " Jul 30 23:01:14.077584 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.077558 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-kube-api-access-rsmnn" (OuterVolumeSpecName: "kube-api-access-rsmnn") pod "e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd" (UID: "e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd"). InnerVolumeSpecName "kube-api-access-rsmnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:01:14.077677 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.077562 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd" (UID: "e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:01:14.091358 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.091330 2554 generic.go:356] "Generic (PLEG): container finished" podID="e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd" containerID="b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3" exitCode=0 Jul 30 23:01:14.091464 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.091379 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-4gzjf" Jul 30 23:01:14.091464 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.091399 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-4gzjf" event={"ID":"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd","Type":"ContainerDied","Data":"b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3"} Jul 30 23:01:14.091464 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.091440 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-4gzjf" event={"ID":"e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd","Type":"ContainerDied","Data":"57bcb147325168eb39d3368b1a7ea970d9d80a7fd557419b704f5d2489dfe0ea"} Jul 30 23:01:14.091572 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.091461 2554 scope.go:117] "RemoveContainer" containerID="b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3" Jul 30 23:01:14.100506 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.100487 2554 scope.go:117] "RemoveContainer" containerID="b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3" Jul 30 23:01:14.100806 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:01:14.100779 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3\": container with ID starting with b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3 not found: ID does not exist" containerID="b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3" Jul 30 23:01:14.100906 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.100816 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3"} err="failed to get container status \"b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3\": rpc error: code = NotFound desc = could not find container \"b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3\": container with ID starting with b5d3880de22d9fc93f3d0bb04b92502a61aa17ade82b56911c73e10db88a00d3 not found: ID does not exist" Jul 30 23:01:14.116203 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.116174 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66ff69c46-4gzjf"] Jul 30 23:01:14.122756 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.122718 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66ff69c46-4gzjf"] Jul 30 23:01:14.176956 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.176909 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsmnn\" (UniqueName: \"kubernetes.io/projected/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-kube-api-access-rsmnn\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:01:14.176956 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.176952 2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd-webhook-cert\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:01:14.970371 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.970338 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66ff69c46-pz992"] Jul 30 23:01:14.970783 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.970769 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd" containerName="manager" Jul 30 23:01:14.970783 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.970784 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd" containerName="manager" Jul 30 23:01:14.970874 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.970865 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd" containerName="manager" Jul 30 23:01:14.975220 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.975202 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:14.978212 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.978195 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 30 23:01:14.979085 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.979067 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 30 23:01:14.979144 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.979105 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l4m8h\"" Jul 30 23:01:14.986522 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:14.986498 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66ff69c46-pz992"] Jul 30 23:01:15.085777 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:15.085716 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m25l\" (UniqueName: \"kubernetes.io/projected/b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254-kube-api-access-8m25l\") pod \"maas-controller-66ff69c46-pz992\" (UID: \"b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254\") " pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:15.085954 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:15.085814 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254-webhook-cert\") pod \"maas-controller-66ff69c46-pz992\" (UID: \"b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254\") " pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:15.186536 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:15.186500 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254-webhook-cert\") pod \"maas-controller-66ff69c46-pz992\" (UID: \"b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254\") " pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:15.186694 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:15.186612 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m25l\" (UniqueName: \"kubernetes.io/projected/b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254-kube-api-access-8m25l\") pod \"maas-controller-66ff69c46-pz992\" (UID: \"b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254\") " pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:15.188997 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:15.188976 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254-webhook-cert\") pod \"maas-controller-66ff69c46-pz992\" (UID: \"b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254\") " pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:15.198656 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:15.198632 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m25l\" (UniqueName: \"kubernetes.io/projected/b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254-kube-api-access-8m25l\") pod \"maas-controller-66ff69c46-pz992\" (UID: \"b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254\") " pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:15.287946 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:15.287847 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:15.439805 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:15.439780 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66ff69c46-pz992"] Jul 30 23:01:15.442635 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:01:15.442610 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35a9fba_bcc5_4bc6_a9bd_6d29d3cbc254.slice/crio-db10cd9424b91d149754c8afedaabd37bc6591e6cf26fba4086c8ab7c523f0dc WatchSource:0}: Error finding container db10cd9424b91d149754c8afedaabd37bc6591e6cf26fba4086c8ab7c523f0dc: Status 404 returned error can't find the container with id db10cd9424b91d149754c8afedaabd37bc6591e6cf26fba4086c8ab7c523f0dc Jul 30 23:01:16.037024 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:16.036936 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd" path="/var/lib/kubelet/pods/e5f3cd3b-5c00-4aa6-9554-ef305a23e7dd/volumes" Jul 30 23:01:16.103591 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:16.103548 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-pz992" event={"ID":"b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254","Type":"ContainerStarted","Data":"2bf99e5139d4d042055f7b1934594a9e6a91eed5fb85c2dc729c9774d0a21fb1"} Jul 30 23:01:16.103771 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:16.103617 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:01:16.103771 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:16.103641 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66ff69c46-pz992" event={"ID":"b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254","Type":"ContainerStarted","Data":"db10cd9424b91d149754c8afedaabd37bc6591e6cf26fba4086c8ab7c523f0dc"} Jul 30 23:01:16.128600 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:16.128534 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66ff69c46-pz992" podStartSLOduration=1.7978916649999999 podStartE2EDuration="2.128511727s" totalImagesPullingTime="330.620062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:01:16.12797983 +0000 UTC m=+822.600443842" watchObservedRunningTime="2026-07-30 23:01:16.128511727 +0000 UTC m=+822.600975742" Jul 30 23:01:27.110796 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:01:27.110761 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66ff69c46-pz992" Jul 30 23:02:34.005369 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:02:34.005291 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:02:34.005896 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:02:34.005696 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:07:34.036383 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:07:34.036351 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:07:34.037993 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:07:34.037971 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:12:34.066050 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:12:34.066016 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:12:34.068631 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:12:34.068606 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:16:39.351166 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.351128 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp"] Jul 30 23:16:39.354988 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.354965 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.357392 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.357362 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c86f68a7-serving-cert\"" Jul 30 23:16:39.364418 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.364382 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp"] Jul 30 23:16:39.392294 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.392259 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsqg4\" (UniqueName: \"kubernetes.io/projected/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-kube-api-access-bsqg4\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.392294 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.392304 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-openshift-service-ca\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.392557 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.392426 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.493498 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.493463 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsqg4\" (UniqueName: \"kubernetes.io/projected/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-kube-api-access-bsqg4\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.493498 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.493511 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-openshift-service-ca\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.493762 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.493573 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.493762 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:16:39.493713 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-serving-cert: secret "maas-api-e2e-ait-c86f68a7-serving-cert" not found Jul 30 23:16:39.493860 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:16:39.493804 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls podName:f8c4be94-de90-4d9d-b8de-1a78cd39dd6c nodeName:}" failed. No retries permitted until 2026-07-30 23:16:39.993783039 +0000 UTC m=+1746.466247029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls") pod "maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" (UID: "f8c4be94-de90-4d9d-b8de-1a78cd39dd6c") : secret "maas-api-e2e-ait-c86f68a7-serving-cert" not found Jul 30 23:16:39.494311 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.494292 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-openshift-service-ca\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.504367 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.504334 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsqg4\" (UniqueName: \"kubernetes.io/projected/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-kube-api-access-bsqg4\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.998411 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:39.998376 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:39.998592 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:16:39.998535 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-serving-cert: secret "maas-api-e2e-ait-c86f68a7-serving-cert" not found Jul 30 23:16:39.998644 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:16:39.998597 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls podName:f8c4be94-de90-4d9d-b8de-1a78cd39dd6c nodeName:}" failed. No retries permitted until 2026-07-30 23:16:40.998582701 +0000 UTC m=+1747.471046691 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls") pod "maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" (UID: "f8c4be94-de90-4d9d-b8de-1a78cd39dd6c") : secret "maas-api-e2e-ait-c86f68a7-serving-cert" not found Jul 30 23:16:41.008863 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.008818 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:41.011317 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.011284 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls\") pod \"maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:41.168180 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.168142 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:41.313277 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.313250 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp"] Jul 30 23:16:41.315293 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:16:41.315264 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c4be94_de90_4d9d_b8de_1a78cd39dd6c.slice/crio-8e628d991971a4b6322e9d46117d3dbf4780c376c6f5cf5fcdb7f17bd96b29d6 WatchSource:0}: Error finding container 8e628d991971a4b6322e9d46117d3dbf4780c376c6f5cf5fcdb7f17bd96b29d6: Status 404 returned error can't find the container with id 8e628d991971a4b6322e9d46117d3dbf4780c376c6f5cf5fcdb7f17bd96b29d6 Jul 30 23:16:41.316653 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.316634 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 23:16:41.766726 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.766692 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" event={"ID":"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c","Type":"ContainerStarted","Data":"cb5af7cee55e8b2e5819e2dc80fde3532957910895333f4a581d4384127a2362"} Jul 30 23:16:41.766915 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.766771 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" event={"ID":"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c","Type":"ContainerStarted","Data":"8e628d991971a4b6322e9d46117d3dbf4780c376c6f5cf5fcdb7f17bd96b29d6"} Jul 30 23:16:41.766915 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.766889 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:16:41.785503 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:41.785453 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" podStartSLOduration=2.427709069 podStartE2EDuration="2.785438139s" totalImagesPullingTime="357.72907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:16:41.783692219 +0000 UTC m=+1748.256156257" watchObservedRunningTime="2026-07-30 23:16:41.785438139 +0000 UTC m=+1748.257902150" Jul 30 23:16:47.777283 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:16:47.777246 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:17:09.734805 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:09.734756 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp"] Jul 30 23:17:09.735304 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:09.735031 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" podUID="f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" containerName="maas-api" containerID="cri-o://cb5af7cee55e8b2e5819e2dc80fde3532957910895333f4a581d4384127a2362" gracePeriod=30 Jul 30 23:17:09.911526 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:09.911487 2554 generic.go:356] "Generic (PLEG): container finished" podID="f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" containerID="cb5af7cee55e8b2e5819e2dc80fde3532957910895333f4a581d4384127a2362" exitCode=0 Jul 30 23:17:09.911658 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:09.911595 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" event={"ID":"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c","Type":"ContainerDied","Data":"cb5af7cee55e8b2e5819e2dc80fde3532957910895333f4a581d4384127a2362"} Jul 30 23:17:09.992984 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:09.992928 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:17:10.070822 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.070788 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsqg4\" (UniqueName: \"kubernetes.io/projected/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-kube-api-access-bsqg4\") pod \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " Jul 30 23:17:10.070822 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.070829 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-openshift-service-ca\") pod \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " Jul 30 23:17:10.071028 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.070885 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls\") pod \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\" (UID: \"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c\") " Jul 30 23:17:10.071312 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.071278 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" (UID: "f8c4be94-de90-4d9d-b8de-1a78cd39dd6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:17:10.074430 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.074405 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" (UID: "f8c4be94-de90-4d9d-b8de-1a78cd39dd6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:17:10.074515 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.074408 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-kube-api-access-bsqg4" (OuterVolumeSpecName: "kube-api-access-bsqg4") pod "f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" (UID: "f8c4be94-de90-4d9d-b8de-1a78cd39dd6c"). InnerVolumeSpecName "kube-api-access-bsqg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:17:10.172434 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.172401 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:17:10.172434 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.172428 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsqg4\" (UniqueName: \"kubernetes.io/projected/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-kube-api-access-bsqg4\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:17:10.172434 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.172441 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:17:10.918825 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.918797 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" Jul 30 23:17:10.919224 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.918801 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp" event={"ID":"f8c4be94-de90-4d9d-b8de-1a78cd39dd6c","Type":"ContainerDied","Data":"8e628d991971a4b6322e9d46117d3dbf4780c376c6f5cf5fcdb7f17bd96b29d6"} Jul 30 23:17:10.919224 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.918947 2554 scope.go:117] "RemoveContainer" containerID="cb5af7cee55e8b2e5819e2dc80fde3532957910895333f4a581d4384127a2362" Jul 30 23:17:10.940976 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.940952 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp"] Jul 30 23:17:10.944685 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:10.944663 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c86f68a7-55d676b69d-vwpmp"] Jul 30 23:17:12.037042 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:12.037011 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" path="/var/lib/kubelet/pods/f8c4be94-de90-4d9d-b8de-1a78cd39dd6c/volumes" Jul 30 23:17:13.549384 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.549343 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l"] Jul 30 23:17:13.550393 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.550368 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" containerName="maas-api" Jul 30 23:17:13.550393 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.550395 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" containerName="maas-api" Jul 30 23:17:13.550624 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.550608 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8c4be94-de90-4d9d-b8de-1a78cd39dd6c" containerName="maas-api" Jul 30 23:17:13.558385 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.558357 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:13.559693 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.559671 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l"] Jul 30 23:17:13.560790 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.560770 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-9d271187-serving-cert\"" Jul 30 23:17:13.708277 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.708239 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w2pt\" (UniqueName: \"kubernetes.io/projected/d051904b-e321-4eb9-9390-596716afedcd-kube-api-access-2w2pt\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:13.708445 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.708288 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:13.708490 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.708464 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d051904b-e321-4eb9-9390-596716afedcd-openshift-service-ca\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:13.809997 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.809912 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d051904b-e321-4eb9-9390-596716afedcd-openshift-service-ca\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:13.809997 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.809970 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w2pt\" (UniqueName: \"kubernetes.io/projected/d051904b-e321-4eb9-9390-596716afedcd-kube-api-access-2w2pt\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:13.809997 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.809996 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:13.810259 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:17:13.810122 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-serving-cert: secret "maas-api-e2e-migrate-9d271187-serving-cert" not found Jul 30 23:17:13.810259 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:17:13.810178 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls podName:d051904b-e321-4eb9-9390-596716afedcd nodeName:}" failed. No retries permitted until 2026-07-30 23:17:14.310161976 +0000 UTC m=+1780.782625967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls") pod "maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" (UID: "d051904b-e321-4eb9-9390-596716afedcd") : secret "maas-api-e2e-migrate-9d271187-serving-cert" not found Jul 30 23:17:13.810570 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.810548 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d051904b-e321-4eb9-9390-596716afedcd-openshift-service-ca\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:13.820462 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:13.820434 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w2pt\" (UniqueName: \"kubernetes.io/projected/d051904b-e321-4eb9-9390-596716afedcd-kube-api-access-2w2pt\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:14.315041 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:14.315002 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:14.317411 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:14.317388 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls\") pod \"maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:14.471060 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:14.471002 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:14.609997 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:14.609967 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l"] Jul 30 23:17:14.611775 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:17:14.611727 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd051904b_e321_4eb9_9390_596716afedcd.slice/crio-19c8124746fbeda63b39f6a07b508791f86a62a21272536f096177b72eb1394f WatchSource:0}: Error finding container 19c8124746fbeda63b39f6a07b508791f86a62a21272536f096177b72eb1394f: Status 404 returned error can't find the container with id 19c8124746fbeda63b39f6a07b508791f86a62a21272536f096177b72eb1394f Jul 30 23:17:14.941996 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:14.941961 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" event={"ID":"d051904b-e321-4eb9-9390-596716afedcd","Type":"ContainerStarted","Data":"19c8124746fbeda63b39f6a07b508791f86a62a21272536f096177b72eb1394f"} Jul 30 23:17:15.948894 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:15.948861 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" event={"ID":"d051904b-e321-4eb9-9390-596716afedcd","Type":"ContainerStarted","Data":"6407c4155ab1cd6ed8ceee81d02349fe2732acfa24444b2fd060a30cc9e99918"} Jul 30 23:17:15.949361 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:15.948946 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:15.964650 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:15.964596 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" podStartSLOduration=2.59518194 podStartE2EDuration="2.964577769s" totalImagesPullingTime="369.395829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:17:15.96386904 +0000 UTC m=+1782.436333053" watchObservedRunningTime="2026-07-30 23:17:15.964577769 +0000 UTC m=+1782.437041783" Jul 30 23:17:21.960269 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:21.960240 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:34.096205 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:34.096162 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:17:34.105115 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:34.105093 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:17:44.967786 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:44.967746 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l"] Jul 30 23:17:44.968221 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:44.967963 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" podUID="d051904b-e321-4eb9-9390-596716afedcd" containerName="maas-api" containerID="cri-o://6407c4155ab1cd6ed8ceee81d02349fe2732acfa24444b2fd060a30cc9e99918" gracePeriod=30 Jul 30 23:17:45.101273 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.101232 2554 generic.go:356] "Generic (PLEG): container finished" podID="d051904b-e321-4eb9-9390-596716afedcd" containerID="6407c4155ab1cd6ed8ceee81d02349fe2732acfa24444b2fd060a30cc9e99918" exitCode=0 Jul 30 23:17:45.101445 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.101307 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" event={"ID":"d051904b-e321-4eb9-9390-596716afedcd","Type":"ContainerDied","Data":"6407c4155ab1cd6ed8ceee81d02349fe2732acfa24444b2fd060a30cc9e99918"} Jul 30 23:17:45.225509 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.225444 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:45.293937 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.293903 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w2pt\" (UniqueName: \"kubernetes.io/projected/d051904b-e321-4eb9-9390-596716afedcd-kube-api-access-2w2pt\") pod \"d051904b-e321-4eb9-9390-596716afedcd\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " Jul 30 23:17:45.294147 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.294013 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls\") pod \"d051904b-e321-4eb9-9390-596716afedcd\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " Jul 30 23:17:45.294147 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.294109 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d051904b-e321-4eb9-9390-596716afedcd-openshift-service-ca\") pod \"d051904b-e321-4eb9-9390-596716afedcd\" (UID: \"d051904b-e321-4eb9-9390-596716afedcd\") " Jul 30 23:17:45.294769 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.294697 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d051904b-e321-4eb9-9390-596716afedcd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d051904b-e321-4eb9-9390-596716afedcd" (UID: "d051904b-e321-4eb9-9390-596716afedcd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:17:45.297402 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.297368 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d051904b-e321-4eb9-9390-596716afedcd-kube-api-access-2w2pt" (OuterVolumeSpecName: "kube-api-access-2w2pt") pod "d051904b-e321-4eb9-9390-596716afedcd" (UID: "d051904b-e321-4eb9-9390-596716afedcd"). InnerVolumeSpecName "kube-api-access-2w2pt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:17:45.297517 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.297478 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d051904b-e321-4eb9-9390-596716afedcd" (UID: "d051904b-e321-4eb9-9390-596716afedcd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:17:45.395978 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.395943 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d051904b-e321-4eb9-9390-596716afedcd-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:17:45.395978 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.395973 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d051904b-e321-4eb9-9390-596716afedcd-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:17:45.395978 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:45.395984 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w2pt\" (UniqueName: \"kubernetes.io/projected/d051904b-e321-4eb9-9390-596716afedcd-kube-api-access-2w2pt\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:17:46.108543 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:46.108514 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" Jul 30 23:17:46.108543 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:46.108524 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l" event={"ID":"d051904b-e321-4eb9-9390-596716afedcd","Type":"ContainerDied","Data":"19c8124746fbeda63b39f6a07b508791f86a62a21272536f096177b72eb1394f"} Jul 30 23:17:46.109051 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:46.108583 2554 scope.go:117] "RemoveContainer" containerID="6407c4155ab1cd6ed8ceee81d02349fe2732acfa24444b2fd060a30cc9e99918" Jul 30 23:17:46.131695 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:46.131662 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l"] Jul 30 23:17:46.135307 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:46.135282 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9d271187-b6b9fdf5-6qs5l"] Jul 30 23:17:48.037307 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:48.037276 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d051904b-e321-4eb9-9390-596716afedcd" path="/var/lib/kubelet/pods/d051904b-e321-4eb9-9390-596716afedcd/volumes" Jul 30 23:17:49.657441 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.657408 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc"] Jul 30 23:17:49.658199 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.658175 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d051904b-e321-4eb9-9390-596716afedcd" containerName="maas-api" Jul 30 23:17:49.658363 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.658340 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d051904b-e321-4eb9-9390-596716afedcd" containerName="maas-api" Jul 30 23:17:49.658507 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.658485 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d051904b-e321-4eb9-9390-596716afedcd" containerName="maas-api" Jul 30 23:17:49.662981 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.662962 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:49.665465 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.665441 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-68124529-serving-cert\"" Jul 30 23:17:49.672718 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.672694 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc"] Jul 30 23:17:49.736926 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.736889 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:49.737246 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.737223 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3f80c8-364a-429b-a6cf-f480caaaae66-openshift-service-ca\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:49.737400 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.737386 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tmpt\" (UniqueName: \"kubernetes.io/projected/db3f80c8-364a-429b-a6cf-f480caaaae66-kube-api-access-6tmpt\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:49.838231 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.838193 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tmpt\" (UniqueName: \"kubernetes.io/projected/db3f80c8-364a-429b-a6cf-f480caaaae66-kube-api-access-6tmpt\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:49.838413 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.838287 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:49.838413 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.838351 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3f80c8-364a-429b-a6cf-f480caaaae66-openshift-service-ca\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:49.838512 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:17:49.838472 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-68124529-serving-cert: secret "maas-api-e2e-ait-68124529-serving-cert" not found Jul 30 23:17:49.838578 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:17:49.838567 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls podName:db3f80c8-364a-429b-a6cf-f480caaaae66 nodeName:}" failed. No retries permitted until 2026-07-30 23:17:50.338543802 +0000 UTC m=+1816.811007797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls") pod "maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" (UID: "db3f80c8-364a-429b-a6cf-f480caaaae66") : secret "maas-api-e2e-ait-68124529-serving-cert" not found Jul 30 23:17:49.839042 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.839026 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3f80c8-364a-429b-a6cf-f480caaaae66-openshift-service-ca\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:49.846979 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:49.846951 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tmpt\" (UniqueName: \"kubernetes.io/projected/db3f80c8-364a-429b-a6cf-f480caaaae66-kube-api-access-6tmpt\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:50.344111 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:50.344078 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:50.346639 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:50.346612 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls\") pod \"maas-api-e2e-ait-68124529-9cc7599b8-8pfpc\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:50.576999 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:50.576964 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:50.924494 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:50.924463 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc"] Jul 30 23:17:50.926219 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:17:50.926191 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3f80c8_364a_429b_a6cf_f480caaaae66.slice/crio-2ab8f2fd9fd7c353b6c8fed47af3f50c609387f21aba64081c6a2a36662110f6 WatchSource:0}: Error finding container 2ab8f2fd9fd7c353b6c8fed47af3f50c609387f21aba64081c6a2a36662110f6: Status 404 returned error can't find the container with id 2ab8f2fd9fd7c353b6c8fed47af3f50c609387f21aba64081c6a2a36662110f6 Jul 30 23:17:51.136947 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:51.136907 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" event={"ID":"db3f80c8-364a-429b-a6cf-f480caaaae66","Type":"ContainerStarted","Data":"2ab8f2fd9fd7c353b6c8fed47af3f50c609387f21aba64081c6a2a36662110f6"} Jul 30 23:17:52.143748 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:52.143690 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" event={"ID":"db3f80c8-364a-429b-a6cf-f480caaaae66","Type":"ContainerStarted","Data":"0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d"} Jul 30 23:17:52.144142 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:52.143779 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:17:52.165022 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:52.164976 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" podStartSLOduration=2.716907684 podStartE2EDuration="3.164962398s" totalImagesPullingTime="448.054714ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:17:52.162869731 +0000 UTC m=+1818.635333746" watchObservedRunningTime="2026-07-30 23:17:52.164962398 +0000 UTC m=+1818.637426409" Jul 30 23:17:58.153898 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:17:58.153866 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:18:20.271249 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.271207 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc"] Jul 30 23:18:20.271871 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.271428 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" podUID="db3f80c8-364a-429b-a6cf-f480caaaae66" containerName="maas-api" containerID="cri-o://0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d" gracePeriod=30 Jul 30 23:18:20.531190 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.531128 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:18:20.618340 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.618302 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tmpt\" (UniqueName: \"kubernetes.io/projected/db3f80c8-364a-429b-a6cf-f480caaaae66-kube-api-access-6tmpt\") pod \"db3f80c8-364a-429b-a6cf-f480caaaae66\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " Jul 30 23:18:20.618340 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.618339 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls\") pod \"db3f80c8-364a-429b-a6cf-f480caaaae66\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " Jul 30 23:18:20.618566 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.618467 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3f80c8-364a-429b-a6cf-f480caaaae66-openshift-service-ca\") pod \"db3f80c8-364a-429b-a6cf-f480caaaae66\" (UID: \"db3f80c8-364a-429b-a6cf-f480caaaae66\") " Jul 30 23:18:20.619033 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.619000 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db3f80c8-364a-429b-a6cf-f480caaaae66-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db3f80c8-364a-429b-a6cf-f480caaaae66" (UID: "db3f80c8-364a-429b-a6cf-f480caaaae66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:18:20.620360 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.620339 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db3f80c8-364a-429b-a6cf-f480caaaae66" (UID: "db3f80c8-364a-429b-a6cf-f480caaaae66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:18:20.620481 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.620461 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db3f80c8-364a-429b-a6cf-f480caaaae66-kube-api-access-6tmpt" (OuterVolumeSpecName: "kube-api-access-6tmpt") pod "db3f80c8-364a-429b-a6cf-f480caaaae66" (UID: "db3f80c8-364a-429b-a6cf-f480caaaae66"). InnerVolumeSpecName "kube-api-access-6tmpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:18:20.719450 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.719404 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3f80c8-364a-429b-a6cf-f480caaaae66-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:18:20.719450 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.719444 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tmpt\" (UniqueName: \"kubernetes.io/projected/db3f80c8-364a-429b-a6cf-f480caaaae66-kube-api-access-6tmpt\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:18:20.719450 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:20.719457 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3f80c8-364a-429b-a6cf-f480caaaae66-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:18:21.295064 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.295029 2554 generic.go:356] "Generic (PLEG): container finished" podID="db3f80c8-364a-429b-a6cf-f480caaaae66" containerID="0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d" exitCode=0 Jul 30 23:18:21.295492 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.295080 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" Jul 30 23:18:21.295492 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.295094 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" event={"ID":"db3f80c8-364a-429b-a6cf-f480caaaae66","Type":"ContainerDied","Data":"0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d"} Jul 30 23:18:21.295492 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.295142 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc" event={"ID":"db3f80c8-364a-429b-a6cf-f480caaaae66","Type":"ContainerDied","Data":"2ab8f2fd9fd7c353b6c8fed47af3f50c609387f21aba64081c6a2a36662110f6"} Jul 30 23:18:21.295492 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.295164 2554 scope.go:117] "RemoveContainer" containerID="0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d" Jul 30 23:18:21.306664 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.306640 2554 scope.go:117] "RemoveContainer" containerID="0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d" Jul 30 23:18:21.306980 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:18:21.306959 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d\": container with ID starting with 0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d not found: ID does not exist" containerID="0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d" Jul 30 23:18:21.307054 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.306988 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d"} err="failed to get container status \"0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d\": rpc error: code = NotFound desc = could not find container \"0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d\": container with ID starting with 0f0a946239b444506ed60bf001523f77495735dac2b2f287694641b0b51ff28d not found: ID does not exist" Jul 30 23:18:21.318279 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.318248 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc"] Jul 30 23:18:21.322914 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:21.322890 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68124529-9cc7599b8-8pfpc"] Jul 30 23:18:22.036820 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:22.036786 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db3f80c8-364a-429b-a6cf-f480caaaae66" path="/var/lib/kubelet/pods/db3f80c8-364a-429b-a6cf-f480caaaae66/volumes" Jul 30 23:18:25.906055 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.906017 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9"] Jul 30 23:18:25.906645 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.906628 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db3f80c8-364a-429b-a6cf-f480caaaae66" containerName="maas-api" Jul 30 23:18:25.906717 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.906646 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="db3f80c8-364a-429b-a6cf-f480caaaae66" containerName="maas-api" Jul 30 23:18:25.907071 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.906802 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="db3f80c8-364a-429b-a6cf-f480caaaae66" containerName="maas-api" Jul 30 23:18:25.911534 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.911511 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:25.914127 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.914094 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c7a3e722-serving-cert\"" Jul 30 23:18:25.916953 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.916924 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9"] Jul 30 23:18:25.961335 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.961297 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:25.961541 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.961353 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csbhp\" (UniqueName: \"kubernetes.io/projected/2a63e066-e2a8-496e-aa11-dec7616c38cb-kube-api-access-csbhp\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:25.961541 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:25.961461 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a63e066-e2a8-496e-aa11-dec7616c38cb-openshift-service-ca\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.061960 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.061924 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.062167 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.061975 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csbhp\" (UniqueName: \"kubernetes.io/projected/2a63e066-e2a8-496e-aa11-dec7616c38cb-kube-api-access-csbhp\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.062167 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.062025 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a63e066-e2a8-496e-aa11-dec7616c38cb-openshift-service-ca\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.062292 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:18:26.062207 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-serving-cert: secret "maas-api-e2e-derive-c7a3e722-serving-cert" not found Jul 30 23:18:26.062332 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:18:26.062298 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls podName:2a63e066-e2a8-496e-aa11-dec7616c38cb nodeName:}" failed. No retries permitted until 2026-07-30 23:18:26.562273336 +0000 UTC m=+1853.034737332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls") pod "maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" (UID: "2a63e066-e2a8-496e-aa11-dec7616c38cb") : secret "maas-api-e2e-derive-c7a3e722-serving-cert" not found Jul 30 23:18:26.062603 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.062584 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a63e066-e2a8-496e-aa11-dec7616c38cb-openshift-service-ca\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.070444 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.070416 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csbhp\" (UniqueName: \"kubernetes.io/projected/2a63e066-e2a8-496e-aa11-dec7616c38cb-kube-api-access-csbhp\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.566353 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.566309 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.568849 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.568815 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls\") pod \"maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.824607 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.824517 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:26.968843 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:26.968812 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9"] Jul 30 23:18:26.970231 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:18:26.970195 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a63e066_e2a8_496e_aa11_dec7616c38cb.slice/crio-41f631d68711473012abfe1256b582ff28c0a4aabcde8e85e7e410ea70f960dc WatchSource:0}: Error finding container 41f631d68711473012abfe1256b582ff28c0a4aabcde8e85e7e410ea70f960dc: Status 404 returned error can't find the container with id 41f631d68711473012abfe1256b582ff28c0a4aabcde8e85e7e410ea70f960dc Jul 30 23:18:27.326490 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:27.326460 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" event={"ID":"2a63e066-e2a8-496e-aa11-dec7616c38cb","Type":"ContainerStarted","Data":"41f631d68711473012abfe1256b582ff28c0a4aabcde8e85e7e410ea70f960dc"} Jul 30 23:18:28.333096 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:28.333053 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" event={"ID":"2a63e066-e2a8-496e-aa11-dec7616c38cb","Type":"ContainerStarted","Data":"e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325"} Jul 30 23:18:28.333546 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:28.333164 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:28.349191 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:28.349142 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" podStartSLOduration=3.002847884 podStartE2EDuration="3.349127825s" totalImagesPullingTime="346.279941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:18:28.348573726 +0000 UTC m=+1854.821037739" watchObservedRunningTime="2026-07-30 23:18:28.349127825 +0000 UTC m=+1854.821591836" Jul 30 23:18:34.345131 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:34.345095 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:56.381359 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.381273 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9"] Jul 30 23:18:56.382099 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.382069 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" podUID="2a63e066-e2a8-496e-aa11-dec7616c38cb" containerName="maas-api" containerID="cri-o://e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325" gracePeriod=30 Jul 30 23:18:56.637070 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.637005 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:56.733917 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.733885 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csbhp\" (UniqueName: \"kubernetes.io/projected/2a63e066-e2a8-496e-aa11-dec7616c38cb-kube-api-access-csbhp\") pod \"2a63e066-e2a8-496e-aa11-dec7616c38cb\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " Jul 30 23:18:56.734108 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.733942 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a63e066-e2a8-496e-aa11-dec7616c38cb-openshift-service-ca\") pod \"2a63e066-e2a8-496e-aa11-dec7616c38cb\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " Jul 30 23:18:56.734108 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.733993 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls\") pod \"2a63e066-e2a8-496e-aa11-dec7616c38cb\" (UID: \"2a63e066-e2a8-496e-aa11-dec7616c38cb\") " Jul 30 23:18:56.734364 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.734317 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a63e066-e2a8-496e-aa11-dec7616c38cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a63e066-e2a8-496e-aa11-dec7616c38cb" (UID: "2a63e066-e2a8-496e-aa11-dec7616c38cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:18:56.736726 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.736683 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a63e066-e2a8-496e-aa11-dec7616c38cb-kube-api-access-csbhp" (OuterVolumeSpecName: "kube-api-access-csbhp") pod "2a63e066-e2a8-496e-aa11-dec7616c38cb" (UID: "2a63e066-e2a8-496e-aa11-dec7616c38cb"). InnerVolumeSpecName "kube-api-access-csbhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:18:56.738018 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.737988 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a63e066-e2a8-496e-aa11-dec7616c38cb" (UID: "2a63e066-e2a8-496e-aa11-dec7616c38cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:18:56.835050 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.835018 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csbhp\" (UniqueName: \"kubernetes.io/projected/2a63e066-e2a8-496e-aa11-dec7616c38cb-kube-api-access-csbhp\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:18:56.835050 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.835044 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a63e066-e2a8-496e-aa11-dec7616c38cb-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:18:56.835050 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:56.835053 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a63e066-e2a8-496e-aa11-dec7616c38cb-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:18:57.480429 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.480388 2554 generic.go:356] "Generic (PLEG): container finished" podID="2a63e066-e2a8-496e-aa11-dec7616c38cb" containerID="e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325" exitCode=0 Jul 30 23:18:57.480429 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.480432 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" Jul 30 23:18:57.480892 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.480469 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" event={"ID":"2a63e066-e2a8-496e-aa11-dec7616c38cb","Type":"ContainerDied","Data":"e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325"} Jul 30 23:18:57.480892 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.480523 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9" event={"ID":"2a63e066-e2a8-496e-aa11-dec7616c38cb","Type":"ContainerDied","Data":"41f631d68711473012abfe1256b582ff28c0a4aabcde8e85e7e410ea70f960dc"} Jul 30 23:18:57.480892 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.480547 2554 scope.go:117] "RemoveContainer" containerID="e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325" Jul 30 23:18:57.490671 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.490650 2554 scope.go:117] "RemoveContainer" containerID="e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325" Jul 30 23:18:57.491036 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:18:57.491007 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325\": container with ID starting with e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325 not found: ID does not exist" containerID="e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325" Jul 30 23:18:57.491153 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.491048 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325"} err="failed to get container status \"e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325\": rpc error: code = NotFound desc = could not find container \"e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325\": container with ID starting with e1a8c7aa87c47d0d19dd4a1261cec7d9d23e1a72f8fbb7bd79424dc04201c325 not found: ID does not exist" Jul 30 23:18:57.511247 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.511210 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9"] Jul 30 23:18:57.525204 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:57.525179 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7a3e722-68f7fdcdc4-mrwl9"] Jul 30 23:18:58.036241 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:18:58.036207 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a63e066-e2a8-496e-aa11-dec7616c38cb" path="/var/lib/kubelet/pods/2a63e066-e2a8-496e-aa11-dec7616c38cb/volumes" Jul 30 23:20:58.719629 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.719541 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt"] Jul 30 23:20:58.722228 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.720002 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a63e066-e2a8-496e-aa11-dec7616c38cb" containerName="maas-api" Jul 30 23:20:58.722228 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.720015 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a63e066-e2a8-496e-aa11-dec7616c38cb" containerName="maas-api" Jul 30 23:20:58.722228 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.720088 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a63e066-e2a8-496e-aa11-dec7616c38cb" containerName="maas-api" Jul 30 23:20:58.723167 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.723151 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:58.725908 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.725883 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7cec9427-serving-cert\"" Jul 30 23:20:58.731904 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.731718 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt"] Jul 30 23:20:58.812549 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.812511 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:58.812722 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.812596 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-openshift-service-ca\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:58.812722 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.812650 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj4ks\" (UniqueName: \"kubernetes.io/projected/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-kube-api-access-sj4ks\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:58.913182 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.913144 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:58.913388 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.913215 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-openshift-service-ca\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:58.913388 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.913250 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj4ks\" (UniqueName: \"kubernetes.io/projected/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-kube-api-access-sj4ks\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:58.913388 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:20:58.913314 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-serving-cert: secret "maas-api-e2e-mt-7cec9427-serving-cert" not found Jul 30 23:20:58.913550 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:20:58.913391 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls podName:e9a04679-0e59-4676-9ff6-f5ae4a54be9d nodeName:}" failed. No retries permitted until 2026-07-30 23:20:59.413371929 +0000 UTC m=+2005.885835919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls") pod "maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" (UID: "e9a04679-0e59-4676-9ff6-f5ae4a54be9d") : secret "maas-api-e2e-mt-7cec9427-serving-cert" not found Jul 30 23:20:58.914046 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.914022 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-openshift-service-ca\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:58.922643 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:58.922613 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj4ks\" (UniqueName: \"kubernetes.io/projected/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-kube-api-access-sj4ks\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:59.417658 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:59.417621 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:59.420130 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:59.420102 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls\") pod \"maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:59.636933 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:59.636900 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:20:59.981774 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:20:59.981643 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt"] Jul 30 23:20:59.984662 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:20:59.984635 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a04679_0e59_4676_9ff6_f5ae4a54be9d.slice/crio-81cef841f24351f8caf1cc170c477a71b148676389dd1e824a8790d893fc87dd WatchSource:0}: Error finding container 81cef841f24351f8caf1cc170c477a71b148676389dd1e824a8790d893fc87dd: Status 404 returned error can't find the container with id 81cef841f24351f8caf1cc170c477a71b148676389dd1e824a8790d893fc87dd Jul 30 23:21:00.090585 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:00.090536 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" event={"ID":"e9a04679-0e59-4676-9ff6-f5ae4a54be9d","Type":"ContainerStarted","Data":"81cef841f24351f8caf1cc170c477a71b148676389dd1e824a8790d893fc87dd"} Jul 30 23:21:01.097398 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:01.097359 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" event={"ID":"e9a04679-0e59-4676-9ff6-f5ae4a54be9d","Type":"ContainerStarted","Data":"7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e"} Jul 30 23:21:01.097862 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:01.097487 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:21:01.113567 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:01.113516 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" podStartSLOduration=2.508983527 podStartE2EDuration="3.113501319s" totalImagesPullingTime="604.517792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:21:01.112367382 +0000 UTC m=+2007.584831414" watchObservedRunningTime="2026-07-30 23:21:01.113501319 +0000 UTC m=+2007.585965331" Jul 30 23:21:07.108402 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:07.108369 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:21:40.456322 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.456282 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt"] Jul 30 23:21:40.456954 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.456582 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" podUID="e9a04679-0e59-4676-9ff6-f5ae4a54be9d" containerName="maas-api" containerID="cri-o://7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e" gracePeriod=30 Jul 30 23:21:40.719518 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.719455 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:21:40.890642 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.890607 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls\") pod \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " Jul 30 23:21:40.890864 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.890694 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj4ks\" (UniqueName: \"kubernetes.io/projected/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-kube-api-access-sj4ks\") pod \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " Jul 30 23:21:40.890864 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.890788 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-openshift-service-ca\") pod \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\" (UID: \"e9a04679-0e59-4676-9ff6-f5ae4a54be9d\") " Jul 30 23:21:40.891163 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.891133 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9a04679-0e59-4676-9ff6-f5ae4a54be9d" (UID: "e9a04679-0e59-4676-9ff6-f5ae4a54be9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:21:40.892905 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.892874 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9a04679-0e59-4676-9ff6-f5ae4a54be9d" (UID: "e9a04679-0e59-4676-9ff6-f5ae4a54be9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:21:40.893000 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.892937 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-kube-api-access-sj4ks" (OuterVolumeSpecName: "kube-api-access-sj4ks") pod "e9a04679-0e59-4676-9ff6-f5ae4a54be9d" (UID: "e9a04679-0e59-4676-9ff6-f5ae4a54be9d"). InnerVolumeSpecName "kube-api-access-sj4ks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:21:40.991772 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.991670 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:21:40.991772 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.991704 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:21:40.991772 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:40.991713 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj4ks\" (UniqueName: \"kubernetes.io/projected/e9a04679-0e59-4676-9ff6-f5ae4a54be9d-kube-api-access-sj4ks\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:21:41.305999 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.305906 2554 generic.go:356] "Generic (PLEG): container finished" podID="e9a04679-0e59-4676-9ff6-f5ae4a54be9d" containerID="7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e" exitCode=0 Jul 30 23:21:41.305999 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.305958 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" Jul 30 23:21:41.305999 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.305974 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" event={"ID":"e9a04679-0e59-4676-9ff6-f5ae4a54be9d","Type":"ContainerDied","Data":"7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e"} Jul 30 23:21:41.306222 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.306027 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt" event={"ID":"e9a04679-0e59-4676-9ff6-f5ae4a54be9d","Type":"ContainerDied","Data":"81cef841f24351f8caf1cc170c477a71b148676389dd1e824a8790d893fc87dd"} Jul 30 23:21:41.306222 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.306057 2554 scope.go:117] "RemoveContainer" containerID="7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e" Jul 30 23:21:41.314937 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.314919 2554 scope.go:117] "RemoveContainer" containerID="7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e" Jul 30 23:21:41.315198 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:21:41.315176 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e\": container with ID starting with 7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e not found: ID does not exist" containerID="7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e" Jul 30 23:21:41.315241 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.315206 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e"} err="failed to get container status \"7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e\": rpc error: code = NotFound desc = could not find container \"7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e\": container with ID starting with 7c9bbfdca69c6ecfcfc175a52dab65f0e463fc95cd8d270df04cd1703683ac3e not found: ID does not exist" Jul 30 23:21:41.329016 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.328993 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt"] Jul 30 23:21:41.332301 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:41.332282 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cec9427-6b9dc49c74-k48zt"] Jul 30 23:21:42.036654 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:21:42.036619 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9a04679-0e59-4676-9ff6-f5ae4a54be9d" path="/var/lib/kubelet/pods/e9a04679-0e59-4676-9ff6-f5ae4a54be9d/volumes" Jul 30 23:22:34.126940 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:22:34.126904 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:22:34.136674 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:22:34.136653 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:24:06.431552 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.431520 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf"] Jul 30 23:24:06.431967 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.431936 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a04679-0e59-4676-9ff6-f5ae4a54be9d" containerName="maas-api" Jul 30 23:24:06.431967 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.431948 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a04679-0e59-4676-9ff6-f5ae4a54be9d" containerName="maas-api" Jul 30 23:24:06.432042 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.432017 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9a04679-0e59-4676-9ff6-f5ae4a54be9d" containerName="maas-api" Jul 30 23:24:06.435091 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.435072 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.438600 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.438573 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d04222a5-serving-cert\"" Jul 30 23:24:06.448271 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.448247 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf"] Jul 30 23:24:06.580157 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.580119 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250ddba6-654b-43be-94ba-c44204ed5715-openshift-service-ca\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.580157 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.580161 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv9rr\" (UniqueName: \"kubernetes.io/projected/250ddba6-654b-43be-94ba-c44204ed5715-kube-api-access-bv9rr\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.580370 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.580184 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250ddba6-654b-43be-94ba-c44204ed5715-maas-api-tls\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.680860 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.680826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250ddba6-654b-43be-94ba-c44204ed5715-maas-api-tls\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.681074 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.680933 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250ddba6-654b-43be-94ba-c44204ed5715-openshift-service-ca\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.681074 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.680955 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv9rr\" (UniqueName: \"kubernetes.io/projected/250ddba6-654b-43be-94ba-c44204ed5715-kube-api-access-bv9rr\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.681686 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.681635 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250ddba6-654b-43be-94ba-c44204ed5715-openshift-service-ca\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.683322 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.683299 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250ddba6-654b-43be-94ba-c44204ed5715-maas-api-tls\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.689720 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.689698 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv9rr\" (UniqueName: \"kubernetes.io/projected/250ddba6-654b-43be-94ba-c44204ed5715-kube-api-access-bv9rr\") pod \"maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.746115 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.746074 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:06.890454 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.890427 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf"] Jul 30 23:24:06.892447 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:24:06.892417 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod250ddba6_654b_43be_94ba_c44204ed5715.slice/crio-322886448f9b4c0551051213561609ab32c154bd3c3d8b700784aab1f1aaefdf WatchSource:0}: Error finding container 322886448f9b4c0551051213561609ab32c154bd3c3d8b700784aab1f1aaefdf: Status 404 returned error can't find the container with id 322886448f9b4c0551051213561609ab32c154bd3c3d8b700784aab1f1aaefdf Jul 30 23:24:06.893816 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:06.893794 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 23:24:07.053210 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:07.053124 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" event={"ID":"250ddba6-654b-43be-94ba-c44204ed5715","Type":"ContainerStarted","Data":"322886448f9b4c0551051213561609ab32c154bd3c3d8b700784aab1f1aaefdf"} Jul 30 23:24:08.059965 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:08.059929 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" event={"ID":"250ddba6-654b-43be-94ba-c44204ed5715","Type":"ContainerStarted","Data":"95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571"} Jul 30 23:24:08.060369 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:08.060050 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:24:08.091210 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:08.091145 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" podStartSLOduration=1.656107187 podStartE2EDuration="2.09112773s" totalImagesPullingTime="435.020543ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:24:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:24:08.089384085 +0000 UTC m=+2194.561848109" watchObservedRunningTime="2026-07-30 23:24:08.09112773 +0000 UTC m=+2194.563591818" Jul 30 23:24:14.070429 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:24:14.070398 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:25:20.463883 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.463841 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf"] Jul 30 23:25:20.464470 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.464101 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" podUID="250ddba6-654b-43be-94ba-c44204ed5715" containerName="maas-api" containerID="cri-o://95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571" gracePeriod=30 Jul 30 23:25:20.738758 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.738699 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:25:20.753498 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.753474 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250ddba6-654b-43be-94ba-c44204ed5715-openshift-service-ca\") pod \"250ddba6-654b-43be-94ba-c44204ed5715\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " Jul 30 23:25:20.753695 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.753518 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv9rr\" (UniqueName: \"kubernetes.io/projected/250ddba6-654b-43be-94ba-c44204ed5715-kube-api-access-bv9rr\") pod \"250ddba6-654b-43be-94ba-c44204ed5715\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " Jul 30 23:25:20.753695 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.753558 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250ddba6-654b-43be-94ba-c44204ed5715-maas-api-tls\") pod \"250ddba6-654b-43be-94ba-c44204ed5715\" (UID: \"250ddba6-654b-43be-94ba-c44204ed5715\") " Jul 30 23:25:20.753835 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.753808 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/250ddba6-654b-43be-94ba-c44204ed5715-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "250ddba6-654b-43be-94ba-c44204ed5715" (UID: "250ddba6-654b-43be-94ba-c44204ed5715"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:25:20.755723 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.755692 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/250ddba6-654b-43be-94ba-c44204ed5715-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "250ddba6-654b-43be-94ba-c44204ed5715" (UID: "250ddba6-654b-43be-94ba-c44204ed5715"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:25:20.755723 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.755719 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/250ddba6-654b-43be-94ba-c44204ed5715-kube-api-access-bv9rr" (OuterVolumeSpecName: "kube-api-access-bv9rr") pod "250ddba6-654b-43be-94ba-c44204ed5715" (UID: "250ddba6-654b-43be-94ba-c44204ed5715"). InnerVolumeSpecName "kube-api-access-bv9rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:25:20.854640 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.854606 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250ddba6-654b-43be-94ba-c44204ed5715-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:25:20.854640 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.854635 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv9rr\" (UniqueName: \"kubernetes.io/projected/250ddba6-654b-43be-94ba-c44204ed5715-kube-api-access-bv9rr\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:25:20.854640 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:20.854645 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250ddba6-654b-43be-94ba-c44204ed5715-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:25:21.426068 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.426027 2554 generic.go:356] "Generic (PLEG): container finished" podID="250ddba6-654b-43be-94ba-c44204ed5715" containerID="95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571" exitCode=0 Jul 30 23:25:21.426249 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.426076 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" Jul 30 23:25:21.426249 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.426114 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" event={"ID":"250ddba6-654b-43be-94ba-c44204ed5715","Type":"ContainerDied","Data":"95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571"} Jul 30 23:25:21.426249 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.426170 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf" event={"ID":"250ddba6-654b-43be-94ba-c44204ed5715","Type":"ContainerDied","Data":"322886448f9b4c0551051213561609ab32c154bd3c3d8b700784aab1f1aaefdf"} Jul 30 23:25:21.426249 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.426202 2554 scope.go:117] "RemoveContainer" containerID="95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571" Jul 30 23:25:21.435850 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.435828 2554 scope.go:117] "RemoveContainer" containerID="95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571" Jul 30 23:25:21.436157 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:25:21.436138 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571\": container with ID starting with 95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571 not found: ID does not exist" containerID="95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571" Jul 30 23:25:21.436235 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.436166 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571"} err="failed to get container status \"95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571\": rpc error: code = NotFound desc = could not find container \"95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571\": container with ID starting with 95dd7feae03456e04e2b2b478250c56200ca81a6d42c8c64fee0443e9e4a6571 not found: ID does not exist" Jul 30 23:25:21.448232 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.448207 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf"] Jul 30 23:25:21.451914 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:21.451892 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d04222a5-5f74cc6cf4-2m9hf"] Jul 30 23:25:22.036288 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:25:22.036256 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="250ddba6-654b-43be-94ba-c44204ed5715" path="/var/lib/kubelet/pods/250ddba6-654b-43be-94ba-c44204ed5715/volumes" Jul 30 23:27:22.666815 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.666777 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm"] Jul 30 23:27:22.667440 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.667421 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="250ddba6-654b-43be-94ba-c44204ed5715" containerName="maas-api" Jul 30 23:27:22.667440 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.667440 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="250ddba6-654b-43be-94ba-c44204ed5715" containerName="maas-api" Jul 30 23:27:22.667590 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.667516 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="250ddba6-654b-43be-94ba-c44204ed5715" containerName="maas-api" Jul 30 23:27:22.670893 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.670874 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:22.673889 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.673871 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2f122f-serving-cert\"" Jul 30 23:27:22.680201 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.680167 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm"] Jul 30 23:27:22.735031 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.734999 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:22.735216 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.735042 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxtbn\" (UniqueName: \"kubernetes.io/projected/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-kube-api-access-dxtbn\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:22.735216 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.735180 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:22.835813 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.835773 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:22.836013 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.835853 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:22.836013 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.835883 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxtbn\" (UniqueName: \"kubernetes.io/projected/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-kube-api-access-dxtbn\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:22.836013 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:27:22.835924 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-serving-cert: secret "maas-api-e2e-inf-a-2f122f-serving-cert" not found Jul 30 23:27:22.836013 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:27:22.836002 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls podName:7868f4a9-6dc4-4586-886e-1b6e0b77aa7c nodeName:}" failed. No retries permitted until 2026-07-30 23:27:23.335985011 +0000 UTC m=+2389.808449021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls") pod "maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" (UID: "7868f4a9-6dc4-4586-886e-1b6e0b77aa7c") : secret "maas-api-e2e-inf-a-2f122f-serving-cert" not found Jul 30 23:27:22.836526 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.836506 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:22.845282 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:22.845249 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxtbn\" (UniqueName: \"kubernetes.io/projected/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-kube-api-access-dxtbn\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:23.339923 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:23.339889 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:23.342350 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:23.342324 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:23.584401 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:23.584371 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:23.736197 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:23.736167 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm"] Jul 30 23:27:23.736873 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:27:23.736847 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7868f4a9_6dc4_4586_886e_1b6e0b77aa7c.slice/crio-f972fea8ba62d7743fb4bc8998458a5246a7b569c35594d62e5ce2bf2542144f WatchSource:0}: Error finding container f972fea8ba62d7743fb4bc8998458a5246a7b569c35594d62e5ce2bf2542144f: Status 404 returned error can't find the container with id f972fea8ba62d7743fb4bc8998458a5246a7b569c35594d62e5ce2bf2542144f Jul 30 23:27:24.039211 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:24.039177 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" event={"ID":"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c","Type":"ContainerStarted","Data":"f972fea8ba62d7743fb4bc8998458a5246a7b569c35594d62e5ce2bf2542144f"} Jul 30 23:27:25.045985 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:25.045951 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" event={"ID":"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c","Type":"ContainerStarted","Data":"9b45c1ec7ef0d28bc573a95d0336b1ba20059f74abb3e64f30b6ad57fbdb7dab"} Jul 30 23:27:25.046418 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:25.046038 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:31.055302 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:31.055275 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:27:31.073503 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:31.073447 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" podStartSLOduration=8.646763756 podStartE2EDuration="9.073425196s" totalImagesPullingTime="426.66144ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:27:25.061966578 +0000 UTC m=+2391.534430591" watchObservedRunningTime="2026-07-30 23:27:31.073425196 +0000 UTC m=+2397.545889199" Jul 30 23:27:34.155716 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:34.155686 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:27:34.174925 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:34.174901 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:27:50.743858 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.743823 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr"] Jul 30 23:27:50.747306 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.747284 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:50.749908 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.749882 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-fcdf3e-serving-cert\"" Jul 30 23:27:50.757549 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.757525 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr"] Jul 30 23:27:50.796441 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.796402 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c2f188-2f37-4b0c-8830-fd47c0b79a97-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:50.796605 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.796449 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2lg8\" (UniqueName: \"kubernetes.io/projected/32c2f188-2f37-4b0c-8830-fd47c0b79a97-kube-api-access-s2lg8\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:50.796605 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.796478 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:50.898012 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.897969 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c2f188-2f37-4b0c-8830-fd47c0b79a97-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:50.898012 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.898014 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2lg8\" (UniqueName: \"kubernetes.io/projected/32c2f188-2f37-4b0c-8830-fd47c0b79a97-kube-api-access-s2lg8\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:50.898294 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.898035 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:50.898294 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:27:50.898124 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-serving-cert: secret "maas-api-e2e-inf-b-fcdf3e-serving-cert" not found Jul 30 23:27:50.898294 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:27:50.898185 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls podName:32c2f188-2f37-4b0c-8830-fd47c0b79a97 nodeName:}" failed. No retries permitted until 2026-07-30 23:27:51.398170079 +0000 UTC m=+2417.870634069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls") pod "maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" (UID: "32c2f188-2f37-4b0c-8830-fd47c0b79a97") : secret "maas-api-e2e-inf-b-fcdf3e-serving-cert" not found Jul 30 23:27:50.898754 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.898709 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c2f188-2f37-4b0c-8830-fd47c0b79a97-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:50.906249 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:50.906223 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2lg8\" (UniqueName: \"kubernetes.io/projected/32c2f188-2f37-4b0c-8830-fd47c0b79a97-kube-api-access-s2lg8\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:51.404178 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:51.404138 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:51.406519 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:51.406495 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls\") pod \"maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:51.660558 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:51.660477 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:51.799002 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:51.798979 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr"] Jul 30 23:27:51.801713 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:27:51.801680 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c2f188_2f37_4b0c_8830_fd47c0b79a97.slice/crio-86743e876536cb7311b79c29b667a16b05eefd4c173e5013aef84d93a6b7f6bb WatchSource:0}: Error finding container 86743e876536cb7311b79c29b667a16b05eefd4c173e5013aef84d93a6b7f6bb: Status 404 returned error can't find the container with id 86743e876536cb7311b79c29b667a16b05eefd4c173e5013aef84d93a6b7f6bb Jul 30 23:27:52.184796 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:52.184768 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" event={"ID":"32c2f188-2f37-4b0c-8830-fd47c0b79a97","Type":"ContainerStarted","Data":"cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433"} Jul 30 23:27:52.184963 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:52.184833 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" event={"ID":"32c2f188-2f37-4b0c-8830-fd47c0b79a97","Type":"ContainerStarted","Data":"86743e876536cb7311b79c29b667a16b05eefd4c173e5013aef84d93a6b7f6bb"} Jul 30 23:27:52.184963 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:52.184864 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:27:52.202456 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:52.202400 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" podStartSLOduration=1.892605586 podStartE2EDuration="2.202379751s" totalImagesPullingTime="309.774165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:27:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:27:52.199701893 +0000 UTC m=+2418.672165904" watchObservedRunningTime="2026-07-30 23:27:52.202379751 +0000 UTC m=+2418.674843765" Jul 30 23:27:58.195646 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:27:58.195571 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:29:25.023714 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.023672 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j"] Jul 30 23:29:25.028005 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.027983 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" Jul 30 23:29:25.031072 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.031038 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-52wcc\"" Jul 30 23:29:25.032429 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.032389 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j"] Jul 30 23:29:25.059249 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.059216 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9790c562-668c-4184-af1b-43fe8be67a8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j\" (UID: \"9790c562-668c-4184-af1b-43fe8be67a8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" Jul 30 23:29:25.160202 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.160164 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9790c562-668c-4184-af1b-43fe8be67a8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j\" (UID: \"9790c562-668c-4184-af1b-43fe8be67a8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" Jul 30 23:29:25.160840 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.160822 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9790c562-668c-4184-af1b-43fe8be67a8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j\" (UID: \"9790c562-668c-4184-af1b-43fe8be67a8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" Jul 30 23:29:25.341517 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.341435 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" Jul 30 23:29:25.476378 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.476353 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j"] Jul 30 23:29:25.478394 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:29:25.478363 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9790c562_668c_4184_af1b_43fe8be67a8a.slice/crio-647f387b1f09bc0c8b2c494d30ccb1032ac80dc068b1321267ec82fe28c23ab6 WatchSource:0}: Error finding container 647f387b1f09bc0c8b2c494d30ccb1032ac80dc068b1321267ec82fe28c23ab6: Status 404 returned error can't find the container with id 647f387b1f09bc0c8b2c494d30ccb1032ac80dc068b1321267ec82fe28c23ab6 Jul 30 23:29:25.480303 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.480285 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 23:29:25.655447 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:25.655362 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" event={"ID":"9790c562-668c-4184-af1b-43fe8be67a8a","Type":"ContainerStarted","Data":"647f387b1f09bc0c8b2c494d30ccb1032ac80dc068b1321267ec82fe28c23ab6"} Jul 30 23:29:28.681873 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:28.681834 2554 generic.go:356] "Generic (PLEG): container finished" podID="9790c562-668c-4184-af1b-43fe8be67a8a" containerID="ecbd984afec25e45f6bd635d685fdf94cafe868b8f42399f6044245f8de2bb66" exitCode=0 Jul 30 23:29:28.682242 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:28.681921 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" event={"ID":"9790c562-668c-4184-af1b-43fe8be67a8a","Type":"ContainerDied","Data":"ecbd984afec25e45f6bd635d685fdf94cafe868b8f42399f6044245f8de2bb66"} Jul 30 23:29:29.831318 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:29.831293 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" Jul 30 23:29:29.906940 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:29.906908 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9790c562-668c-4184-af1b-43fe8be67a8a-maas-api-service-ca\") pod \"9790c562-668c-4184-af1b-43fe8be67a8a\" (UID: \"9790c562-668c-4184-af1b-43fe8be67a8a\") " Jul 30 23:29:29.907282 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:29.907257 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9790c562-668c-4184-af1b-43fe8be67a8a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9790c562-668c-4184-af1b-43fe8be67a8a" (UID: "9790c562-668c-4184-af1b-43fe8be67a8a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:29:30.008491 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:30.008411 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9790c562-668c-4184-af1b-43fe8be67a8a-maas-api-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:29:30.693248 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:30.693217 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" event={"ID":"9790c562-668c-4184-af1b-43fe8be67a8a","Type":"ContainerDied","Data":"647f387b1f09bc0c8b2c494d30ccb1032ac80dc068b1321267ec82fe28c23ab6"} Jul 30 23:29:30.693423 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:30.693264 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="647f387b1f09bc0c8b2c494d30ccb1032ac80dc068b1321267ec82fe28c23ab6" Jul 30 23:29:30.693423 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:30.693229 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j" Jul 30 23:29:35.593961 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.593926 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm"] Jul 30 23:29:35.594368 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.594182 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" podUID="7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" containerName="maas-api" containerID="cri-o://9b45c1ec7ef0d28bc573a95d0336b1ba20059f74abb3e64f30b6ad57fbdb7dab" gracePeriod=30 Jul 30 23:29:35.720132 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.720102 2554 generic.go:356] "Generic (PLEG): container finished" podID="7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" containerID="9b45c1ec7ef0d28bc573a95d0336b1ba20059f74abb3e64f30b6ad57fbdb7dab" exitCode=0 Jul 30 23:29:35.720274 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.720154 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" event={"ID":"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c","Type":"ContainerDied","Data":"9b45c1ec7ef0d28bc573a95d0336b1ba20059f74abb3e64f30b6ad57fbdb7dab"} Jul 30 23:29:35.806751 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.806697 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j"] Jul 30 23:29:35.809357 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.809332 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2f122f-m8k8j"] Jul 30 23:29:35.861979 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.861955 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:29:35.958863 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.958837 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls\") pod \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " Jul 30 23:29:35.959017 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.958887 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-openshift-service-ca\") pod \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " Jul 30 23:29:35.959017 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.958914 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxtbn\" (UniqueName: \"kubernetes.io/projected/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-kube-api-access-dxtbn\") pod \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\" (UID: \"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c\") " Jul 30 23:29:35.959300 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.959269 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" (UID: "7868f4a9-6dc4-4586-886e-1b6e0b77aa7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:29:35.960955 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.960929 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" (UID: "7868f4a9-6dc4-4586-886e-1b6e0b77aa7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:29:35.961180 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:35.961153 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-kube-api-access-dxtbn" (OuterVolumeSpecName: "kube-api-access-dxtbn") pod "7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" (UID: "7868f4a9-6dc4-4586-886e-1b6e0b77aa7c"). InnerVolumeSpecName "kube-api-access-dxtbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:29:36.036640 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.036609 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9790c562-668c-4184-af1b-43fe8be67a8a" path="/var/lib/kubelet/pods/9790c562-668c-4184-af1b-43fe8be67a8a/volumes" Jul 30 23:29:36.060341 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.060317 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:29:36.060341 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.060339 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:29:36.060463 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.060351 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxtbn\" (UniqueName: \"kubernetes.io/projected/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c-kube-api-access-dxtbn\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:29:36.727282 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.727253 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" Jul 30 23:29:36.727823 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.727295 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm" event={"ID":"7868f4a9-6dc4-4586-886e-1b6e0b77aa7c","Type":"ContainerDied","Data":"f972fea8ba62d7743fb4bc8998458a5246a7b569c35594d62e5ce2bf2542144f"} Jul 30 23:29:36.727823 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.727356 2554 scope.go:117] "RemoveContainer" containerID="9b45c1ec7ef0d28bc573a95d0336b1ba20059f74abb3e64f30b6ad57fbdb7dab" Jul 30 23:29:36.743960 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.743927 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm"] Jul 30 23:29:36.747706 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:36.747685 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f122f-559b779f85-cgqlm"] Jul 30 23:29:38.036479 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:38.036445 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" path="/var/lib/kubelet/pods/7868f4a9-6dc4-4586-886e-1b6e0b77aa7c/volumes" Jul 30 23:29:42.784678 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.784646 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf"] Jul 30 23:29:42.785101 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.785088 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9790c562-668c-4184-af1b-43fe8be67a8a" containerName="revoke-keys" Jul 30 23:29:42.785101 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.785102 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9790c562-668c-4184-af1b-43fe8be67a8a" containerName="revoke-keys" Jul 30 23:29:42.785239 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.785118 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" containerName="maas-api" Jul 30 23:29:42.785239 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.785126 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" containerName="maas-api" Jul 30 23:29:42.785239 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.785193 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7868f4a9-6dc4-4586-886e-1b6e0b77aa7c" containerName="maas-api" Jul 30 23:29:42.785239 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.785209 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9790c562-668c-4184-af1b-43fe8be67a8a" containerName="revoke-keys" Jul 30 23:29:42.788420 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.788403 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" Jul 30 23:29:42.790809 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.790788 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-52wcc\"" Jul 30 23:29:42.796159 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.795598 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf"] Jul 30 23:29:42.923356 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:42.923320 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c84047-1afc-4887-8312-73dc2a6e769b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf\" (UID: \"55c84047-1afc-4887-8312-73dc2a6e769b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" Jul 30 23:29:43.024435 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:43.024403 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c84047-1afc-4887-8312-73dc2a6e769b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf\" (UID: \"55c84047-1afc-4887-8312-73dc2a6e769b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" Jul 30 23:29:43.025044 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:43.025024 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c84047-1afc-4887-8312-73dc2a6e769b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf\" (UID: \"55c84047-1afc-4887-8312-73dc2a6e769b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" Jul 30 23:29:43.101993 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:43.101924 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" Jul 30 23:29:43.241633 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:43.241605 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf"] Jul 30 23:29:43.244032 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:29:43.244005 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c84047_1afc_4887_8312_73dc2a6e769b.slice/crio-5da0b0c25182ced00fb915823ea8d63a71e1b9471d283115948d6e84480f9d5f WatchSource:0}: Error finding container 5da0b0c25182ced00fb915823ea8d63a71e1b9471d283115948d6e84480f9d5f: Status 404 returned error can't find the container with id 5da0b0c25182ced00fb915823ea8d63a71e1b9471d283115948d6e84480f9d5f Jul 30 23:29:43.765117 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:43.765023 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" event={"ID":"55c84047-1afc-4887-8312-73dc2a6e769b","Type":"ContainerStarted","Data":"3301339568fcc77c0e23c183fc926129346386c04fdc69a70ba32976627d0a03"} Jul 30 23:29:43.765117 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:43.765079 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" event={"ID":"55c84047-1afc-4887-8312-73dc2a6e769b","Type":"ContainerStarted","Data":"5da0b0c25182ced00fb915823ea8d63a71e1b9471d283115948d6e84480f9d5f"} Jul 30 23:29:43.780912 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:43.780865 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" podStartSLOduration=1.780849351 podStartE2EDuration="1.780849351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:29:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:29:43.778909826 +0000 UTC m=+2530.251373868" watchObservedRunningTime="2026-07-30 23:29:43.780849351 +0000 UTC m=+2530.253313363" Jul 30 23:29:44.772825 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:44.772782 2554 generic.go:356] "Generic (PLEG): container finished" podID="55c84047-1afc-4887-8312-73dc2a6e769b" containerID="3301339568fcc77c0e23c183fc926129346386c04fdc69a70ba32976627d0a03" exitCode=0 Jul 30 23:29:44.773274 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:44.772865 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" event={"ID":"55c84047-1afc-4887-8312-73dc2a6e769b","Type":"ContainerDied","Data":"3301339568fcc77c0e23c183fc926129346386c04fdc69a70ba32976627d0a03"} Jul 30 23:29:45.918601 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:45.918576 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" Jul 30 23:29:46.052558 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:46.052476 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c84047-1afc-4887-8312-73dc2a6e769b-maas-api-service-ca\") pod \"55c84047-1afc-4887-8312-73dc2a6e769b\" (UID: \"55c84047-1afc-4887-8312-73dc2a6e769b\") " Jul 30 23:29:46.052835 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:46.052812 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55c84047-1afc-4887-8312-73dc2a6e769b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55c84047-1afc-4887-8312-73dc2a6e769b" (UID: "55c84047-1afc-4887-8312-73dc2a6e769b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:29:46.153499 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:46.153452 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c84047-1afc-4887-8312-73dc2a6e769b-maas-api-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:29:46.785376 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:46.785344 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" Jul 30 23:29:46.785540 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:46.785389 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf" event={"ID":"55c84047-1afc-4887-8312-73dc2a6e769b","Type":"ContainerDied","Data":"5da0b0c25182ced00fb915823ea8d63a71e1b9471d283115948d6e84480f9d5f"} Jul 30 23:29:46.785540 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:46.785433 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5da0b0c25182ced00fb915823ea8d63a71e1b9471d283115948d6e84480f9d5f" Jul 30 23:29:53.208636 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.208600 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr"] Jul 30 23:29:53.209231 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.208823 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" podUID="32c2f188-2f37-4b0c-8830-fd47c0b79a97" containerName="maas-api" containerID="cri-o://cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433" gracePeriod=30 Jul 30 23:29:53.456252 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.456221 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf"] Jul 30 23:29:53.459219 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.459161 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fcdf3e-2bswf"] Jul 30 23:29:53.474027 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.474006 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:29:53.620965 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.620933 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls\") pod \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " Jul 30 23:29:53.621114 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.621080 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c2f188-2f37-4b0c-8830-fd47c0b79a97-openshift-service-ca\") pod \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " Jul 30 23:29:53.621114 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.621110 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2lg8\" (UniqueName: \"kubernetes.io/projected/32c2f188-2f37-4b0c-8830-fd47c0b79a97-kube-api-access-s2lg8\") pod \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\" (UID: \"32c2f188-2f37-4b0c-8830-fd47c0b79a97\") " Jul 30 23:29:53.621422 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.621393 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c2f188-2f37-4b0c-8830-fd47c0b79a97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "32c2f188-2f37-4b0c-8830-fd47c0b79a97" (UID: "32c2f188-2f37-4b0c-8830-fd47c0b79a97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:29:53.622971 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.622947 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "32c2f188-2f37-4b0c-8830-fd47c0b79a97" (UID: "32c2f188-2f37-4b0c-8830-fd47c0b79a97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:29:53.623127 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.623112 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c2f188-2f37-4b0c-8830-fd47c0b79a97-kube-api-access-s2lg8" (OuterVolumeSpecName: "kube-api-access-s2lg8") pod "32c2f188-2f37-4b0c-8830-fd47c0b79a97" (UID: "32c2f188-2f37-4b0c-8830-fd47c0b79a97"). InnerVolumeSpecName "kube-api-access-s2lg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:29:53.722106 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.722022 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c2f188-2f37-4b0c-8830-fd47c0b79a97-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:29:53.722106 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.722051 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2lg8\" (UniqueName: \"kubernetes.io/projected/32c2f188-2f37-4b0c-8830-fd47c0b79a97-kube-api-access-s2lg8\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:29:53.722106 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.722063 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32c2f188-2f37-4b0c-8830-fd47c0b79a97-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:29:53.826360 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.826324 2554 generic.go:356] "Generic (PLEG): container finished" podID="32c2f188-2f37-4b0c-8830-fd47c0b79a97" containerID="cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433" exitCode=0 Jul 30 23:29:53.826512 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.826374 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" Jul 30 23:29:53.826512 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.826414 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" event={"ID":"32c2f188-2f37-4b0c-8830-fd47c0b79a97","Type":"ContainerDied","Data":"cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433"} Jul 30 23:29:53.826512 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.826470 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr" event={"ID":"32c2f188-2f37-4b0c-8830-fd47c0b79a97","Type":"ContainerDied","Data":"86743e876536cb7311b79c29b667a16b05eefd4c173e5013aef84d93a6b7f6bb"} Jul 30 23:29:53.826512 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.826491 2554 scope.go:117] "RemoveContainer" containerID="cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433" Jul 30 23:29:53.836662 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.836645 2554 scope.go:117] "RemoveContainer" containerID="cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433" Jul 30 23:29:53.836941 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:29:53.836920 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433\": container with ID starting with cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433 not found: ID does not exist" containerID="cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433" Jul 30 23:29:53.837005 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.836948 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433"} err="failed to get container status \"cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433\": rpc error: code = NotFound desc = could not find container \"cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433\": container with ID starting with cf8806a2298c63b4593b25e8e14854710871d3f8ee2f155c12284fe8dd871433 not found: ID does not exist" Jul 30 23:29:53.848922 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.848902 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr"] Jul 30 23:29:53.855744 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:53.855719 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fcdf3e-d7848d5d4-6hddr"] Jul 30 23:29:54.037394 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:54.037315 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32c2f188-2f37-4b0c-8830-fd47c0b79a97" path="/var/lib/kubelet/pods/32c2f188-2f37-4b0c-8830-fd47c0b79a97/volumes" Jul 30 23:29:54.037672 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:29:54.037659 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55c84047-1afc-4887-8312-73dc2a6e769b" path="/var/lib/kubelet/pods/55c84047-1afc-4887-8312-73dc2a6e769b/volumes" Jul 30 23:30:05.376650 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.376226 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt"] Jul 30 23:30:05.377287 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.376723 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32c2f188-2f37-4b0c-8830-fd47c0b79a97" containerName="maas-api" Jul 30 23:30:05.377287 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.376753 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c2f188-2f37-4b0c-8830-fd47c0b79a97" containerName="maas-api" Jul 30 23:30:05.377287 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.376787 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55c84047-1afc-4887-8312-73dc2a6e769b" containerName="revoke-keys" Jul 30 23:30:05.377287 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.376793 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="55c84047-1afc-4887-8312-73dc2a6e769b" containerName="revoke-keys" Jul 30 23:30:05.377287 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.376877 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="32c2f188-2f37-4b0c-8830-fd47c0b79a97" containerName="maas-api" Jul 30 23:30:05.377287 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.376890 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="55c84047-1afc-4887-8312-73dc2a6e769b" containerName="revoke-keys" Jul 30 23:30:05.383999 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.383974 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:05.386379 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.386358 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-0ff11c-serving-cert\"" Jul 30 23:30:05.391552 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.391523 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt"] Jul 30 23:30:05.526556 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.526518 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc82f\" (UniqueName: \"kubernetes.io/projected/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-kube-api-access-tc82f\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:05.526774 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.526586 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-openshift-service-ca\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:05.526774 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.526612 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:05.628135 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.628038 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc82f\" (UniqueName: \"kubernetes.io/projected/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-kube-api-access-tc82f\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:05.628135 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.628093 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-openshift-service-ca\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:05.628135 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.628110 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:05.628460 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:30:05.628239 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-serving-cert: secret "maas-api-e2e-api-a-0ff11c-serving-cert" not found Jul 30 23:30:05.628460 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:30:05.628322 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls podName:50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f nodeName:}" failed. No retries permitted until 2026-07-30 23:30:06.128306727 +0000 UTC m=+2552.600770718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls") pod "maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" (UID: "50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f") : secret "maas-api-e2e-api-a-0ff11c-serving-cert" not found Jul 30 23:30:05.628794 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.628774 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-openshift-service-ca\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:05.639323 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:05.639294 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc82f\" (UniqueName: \"kubernetes.io/projected/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-kube-api-access-tc82f\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:06.133893 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:06.133856 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:06.136620 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:06.136590 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls\") pod \"maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:06.300793 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:06.300726 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:06.450121 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:06.450095 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt"] Jul 30 23:30:06.453410 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:30:06.453386 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f8acc5_49bf_4a6d_982b_d6d0de4b8e0f.slice/crio-598598bcadd8c42fc46ec0d9e3367739d682fe2a8f564e734e3d07473aa5414f WatchSource:0}: Error finding container 598598bcadd8c42fc46ec0d9e3367739d682fe2a8f564e734e3d07473aa5414f: Status 404 returned error can't find the container with id 598598bcadd8c42fc46ec0d9e3367739d682fe2a8f564e734e3d07473aa5414f Jul 30 23:30:06.898953 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:06.898917 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" event={"ID":"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f","Type":"ContainerStarted","Data":"598598bcadd8c42fc46ec0d9e3367739d682fe2a8f564e734e3d07473aa5414f"} Jul 30 23:30:07.905802 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:07.905755 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" event={"ID":"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f","Type":"ContainerStarted","Data":"fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc"} Jul 30 23:30:07.906201 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:07.905830 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:07.924703 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:07.924647 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" podStartSLOduration=2.292369741 podStartE2EDuration="2.924630012s" totalImagesPullingTime="632.260271ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:30:07.92318035 +0000 UTC m=+2554.395644363" watchObservedRunningTime="2026-07-30 23:30:07.924630012 +0000 UTC m=+2554.397094024" Jul 30 23:30:13.915720 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:13.915682 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:30:34.560025 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.559943 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84"] Jul 30 23:30:34.565578 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.565554 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:34.567959 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.567932 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-13c529-serving-cert\"" Jul 30 23:30:34.572993 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.572971 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84"] Jul 30 23:30:34.678492 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.678450 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:34.678492 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.678494 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79456e55-d5f6-4965-8790-d8190e9af42e-openshift-service-ca\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:34.678717 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.678516 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5vhh\" (UniqueName: \"kubernetes.io/projected/79456e55-d5f6-4965-8790-d8190e9af42e-kube-api-access-k5vhh\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:34.780007 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.779970 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:34.780218 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.780019 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79456e55-d5f6-4965-8790-d8190e9af42e-openshift-service-ca\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:34.780218 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.780051 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5vhh\" (UniqueName: \"kubernetes.io/projected/79456e55-d5f6-4965-8790-d8190e9af42e-kube-api-access-k5vhh\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:34.780218 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:30:34.780122 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-serving-cert: secret "maas-api-e2e-api-b-13c529-serving-cert" not found Jul 30 23:30:34.780218 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:30:34.780205 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls podName:79456e55-d5f6-4965-8790-d8190e9af42e nodeName:}" failed. No retries permitted until 2026-07-30 23:30:35.280186694 +0000 UTC m=+2581.752650703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls") pod "maas-api-e2e-api-b-13c529-64988b5d97-c4c84" (UID: "79456e55-d5f6-4965-8790-d8190e9af42e") : secret "maas-api-e2e-api-b-13c529-serving-cert" not found Jul 30 23:30:34.780762 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.780719 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79456e55-d5f6-4965-8790-d8190e9af42e-openshift-service-ca\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:34.788665 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:34.788634 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5vhh\" (UniqueName: \"kubernetes.io/projected/79456e55-d5f6-4965-8790-d8190e9af42e-kube-api-access-k5vhh\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:35.285014 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:35.284976 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:35.287533 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:35.287507 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls\") pod \"maas-api-e2e-api-b-13c529-64988b5d97-c4c84\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:35.480878 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:35.480837 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:35.626974 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:35.626941 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84"] Jul 30 23:30:35.628941 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:30:35.628915 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79456e55_d5f6_4965_8790_d8190e9af42e.slice/crio-11b1f11c05bf4c9583de646a8a42b1510cb5144aa2b70d85e6a5a66a1c37bd23 WatchSource:0}: Error finding container 11b1f11c05bf4c9583de646a8a42b1510cb5144aa2b70d85e6a5a66a1c37bd23: Status 404 returned error can't find the container with id 11b1f11c05bf4c9583de646a8a42b1510cb5144aa2b70d85e6a5a66a1c37bd23 Jul 30 23:30:36.053092 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:36.053058 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" event={"ID":"79456e55-d5f6-4965-8790-d8190e9af42e","Type":"ContainerStarted","Data":"938b8c9338d5471a4f2c454be87a183298c0d7c4dca63fbf6ab4e42c0cc6546e"} Jul 30 23:30:36.053192 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:36.053120 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" event={"ID":"79456e55-d5f6-4965-8790-d8190e9af42e","Type":"ContainerStarted","Data":"11b1f11c05bf4c9583de646a8a42b1510cb5144aa2b70d85e6a5a66a1c37bd23"} Jul 30 23:30:36.053192 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:36.053160 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:30:36.070599 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:36.070558 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" podStartSLOduration=1.71955234 podStartE2EDuration="2.070544091s" totalImagesPullingTime="350.991751ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:30:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:30:36.068298783 +0000 UTC m=+2582.540762794" watchObservedRunningTime="2026-07-30 23:30:36.070544091 +0000 UTC m=+2582.543008102" Jul 30 23:30:42.063528 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:30:42.063496 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:31:16.756335 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:16.756302 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt"] Jul 30 23:31:16.756920 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:16.756561 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" podUID="50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" containerName="maas-api" containerID="cri-o://fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc" gracePeriod=30 Jul 30 23:31:17.016219 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.016160 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:31:17.061291 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.061264 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls\") pod \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " Jul 30 23:31:17.061449 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.061366 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc82f\" (UniqueName: \"kubernetes.io/projected/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-kube-api-access-tc82f\") pod \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " Jul 30 23:31:17.061449 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.061400 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-openshift-service-ca\") pod \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\" (UID: \"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f\") " Jul 30 23:31:17.062010 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.061982 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" (UID: "50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:31:17.063534 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.063509 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" (UID: "50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:31:17.064312 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.064293 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-kube-api-access-tc82f" (OuterVolumeSpecName: "kube-api-access-tc82f") pod "50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" (UID: "50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f"). InnerVolumeSpecName "kube-api-access-tc82f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:31:17.162723 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.162687 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc82f\" (UniqueName: \"kubernetes.io/projected/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-kube-api-access-tc82f\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:31:17.162723 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.162716 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:31:17.162723 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.162725 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:31:17.262050 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.262015 2554 generic.go:356] "Generic (PLEG): container finished" podID="50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" containerID="fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc" exitCode=0 Jul 30 23:31:17.262226 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.262064 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" Jul 30 23:31:17.262226 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.262081 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" event={"ID":"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f","Type":"ContainerDied","Data":"fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc"} Jul 30 23:31:17.262226 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.262131 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt" event={"ID":"50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f","Type":"ContainerDied","Data":"598598bcadd8c42fc46ec0d9e3367739d682fe2a8f564e734e3d07473aa5414f"} Jul 30 23:31:17.262226 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.262164 2554 scope.go:117] "RemoveContainer" containerID="fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc" Jul 30 23:31:17.272538 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.272517 2554 scope.go:117] "RemoveContainer" containerID="fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc" Jul 30 23:31:17.272820 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:31:17.272800 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc\": container with ID starting with fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc not found: ID does not exist" containerID="fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc" Jul 30 23:31:17.272894 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.272827 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc"} err="failed to get container status \"fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc\": rpc error: code = NotFound desc = could not find container \"fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc\": container with ID starting with fe6a8b144b569f66ee1261bc3a5c84998bbf7d2bd328a4b9edc99c41f696afdc not found: ID does not exist" Jul 30 23:31:17.285141 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.285117 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt"] Jul 30 23:31:17.289496 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:17.289474 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0ff11c-5867fb447b-hbxmt"] Jul 30 23:31:18.037905 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:18.037874 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" path="/var/lib/kubelet/pods/50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f/volumes" Jul 30 23:31:23.817011 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:23.816972 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2"] Jul 30 23:31:23.817455 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:23.817405 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" containerName="maas-api" Jul 30 23:31:23.817455 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:23.817416 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" containerName="maas-api" Jul 30 23:31:23.817533 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:23.817493 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f8acc5-49bf-4a6d-982b-d6d0de4b8e0f" containerName="maas-api" Jul 30 23:31:23.821894 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:23.821875 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" Jul 30 23:31:23.827406 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:23.827382 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-52wcc\"" Jul 30 23:31:23.832302 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:23.832276 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2"] Jul 30 23:31:23.924683 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:23.924646 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82f1c3d-448f-4bef-86b6-b5bb68f83c5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-13c529-24rl2\" (UID: \"a82f1c3d-448f-4bef-86b6-b5bb68f83c5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" Jul 30 23:31:24.025685 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:24.025651 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82f1c3d-448f-4bef-86b6-b5bb68f83c5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-13c529-24rl2\" (UID: \"a82f1c3d-448f-4bef-86b6-b5bb68f83c5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" Jul 30 23:31:24.026280 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:24.026259 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82f1c3d-448f-4bef-86b6-b5bb68f83c5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-13c529-24rl2\" (UID: \"a82f1c3d-448f-4bef-86b6-b5bb68f83c5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" Jul 30 23:31:24.132678 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:24.132597 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" Jul 30 23:31:24.273934 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:24.273911 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2"] Jul 30 23:31:24.276923 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:31:24.276895 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82f1c3d_448f_4bef_86b6_b5bb68f83c5a.slice/crio-e82a38a7241dca8bf9934a7372d4b0746a83fd30f24657725731332b4903a13e WatchSource:0}: Error finding container e82a38a7241dca8bf9934a7372d4b0746a83fd30f24657725731332b4903a13e: Status 404 returned error can't find the container with id e82a38a7241dca8bf9934a7372d4b0746a83fd30f24657725731332b4903a13e Jul 30 23:31:24.301729 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:24.301700 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" event={"ID":"a82f1c3d-448f-4bef-86b6-b5bb68f83c5a","Type":"ContainerStarted","Data":"e82a38a7241dca8bf9934a7372d4b0746a83fd30f24657725731332b4903a13e"} Jul 30 23:31:25.308434 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:25.308399 2554 generic.go:356] "Generic (PLEG): container finished" podID="a82f1c3d-448f-4bef-86b6-b5bb68f83c5a" containerID="21fbb9884d2f8f3a699779c783f458666565dd7cb6b8d0ae0697c9925d102cc6" exitCode=0 Jul 30 23:31:25.308895 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:25.308489 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" event={"ID":"a82f1c3d-448f-4bef-86b6-b5bb68f83c5a","Type":"ContainerDied","Data":"21fbb9884d2f8f3a699779c783f458666565dd7cb6b8d0ae0697c9925d102cc6"} Jul 30 23:31:26.460213 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:26.460188 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" Jul 30 23:31:26.548577 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:26.548546 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82f1c3d-448f-4bef-86b6-b5bb68f83c5a-maas-api-service-ca\") pod \"a82f1c3d-448f-4bef-86b6-b5bb68f83c5a\" (UID: \"a82f1c3d-448f-4bef-86b6-b5bb68f83c5a\") " Jul 30 23:31:26.548936 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:26.548914 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a82f1c3d-448f-4bef-86b6-b5bb68f83c5a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a82f1c3d-448f-4bef-86b6-b5bb68f83c5a" (UID: "a82f1c3d-448f-4bef-86b6-b5bb68f83c5a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:31:26.649195 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:26.649113 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82f1c3d-448f-4bef-86b6-b5bb68f83c5a-maas-api-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:31:27.320729 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:27.320696 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" Jul 30 23:31:27.320933 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:27.320729 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2" event={"ID":"a82f1c3d-448f-4bef-86b6-b5bb68f83c5a","Type":"ContainerDied","Data":"e82a38a7241dca8bf9934a7372d4b0746a83fd30f24657725731332b4903a13e"} Jul 30 23:31:27.320933 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:27.320837 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e82a38a7241dca8bf9934a7372d4b0746a83fd30f24657725731332b4903a13e" Jul 30 23:31:39.250052 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.250009 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84"] Jul 30 23:31:39.250611 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.250230 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" podUID="79456e55-d5f6-4965-8790-d8190e9af42e" containerName="maas-api" containerID="cri-o://938b8c9338d5471a4f2c454be87a183298c0d7c4dca63fbf6ab4e42c0cc6546e" gracePeriod=30 Jul 30 23:31:39.386206 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.386167 2554 generic.go:356] "Generic (PLEG): container finished" podID="79456e55-d5f6-4965-8790-d8190e9af42e" containerID="938b8c9338d5471a4f2c454be87a183298c0d7c4dca63fbf6ab4e42c0cc6546e" exitCode=0 Jul 30 23:31:39.386387 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.386245 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" event={"ID":"79456e55-d5f6-4965-8790-d8190e9af42e","Type":"ContainerDied","Data":"938b8c9338d5471a4f2c454be87a183298c0d7c4dca63fbf6ab4e42c0cc6546e"} Jul 30 23:31:39.461320 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.461285 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2"] Jul 30 23:31:39.464462 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.464433 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-13c529-24rl2"] Jul 30 23:31:39.508258 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.508205 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:31:39.567214 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.567183 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79456e55-d5f6-4965-8790-d8190e9af42e-openshift-service-ca\") pod \"79456e55-d5f6-4965-8790-d8190e9af42e\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " Jul 30 23:31:39.567403 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.567250 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5vhh\" (UniqueName: \"kubernetes.io/projected/79456e55-d5f6-4965-8790-d8190e9af42e-kube-api-access-k5vhh\") pod \"79456e55-d5f6-4965-8790-d8190e9af42e\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " Jul 30 23:31:39.567403 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.567308 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls\") pod \"79456e55-d5f6-4965-8790-d8190e9af42e\" (UID: \"79456e55-d5f6-4965-8790-d8190e9af42e\") " Jul 30 23:31:39.567539 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.567511 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79456e55-d5f6-4965-8790-d8190e9af42e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79456e55-d5f6-4965-8790-d8190e9af42e" (UID: "79456e55-d5f6-4965-8790-d8190e9af42e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:31:39.569385 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.569355 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79456e55-d5f6-4965-8790-d8190e9af42e" (UID: "79456e55-d5f6-4965-8790-d8190e9af42e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:31:39.569489 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.569394 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79456e55-d5f6-4965-8790-d8190e9af42e-kube-api-access-k5vhh" (OuterVolumeSpecName: "kube-api-access-k5vhh") pod "79456e55-d5f6-4965-8790-d8190e9af42e" (UID: "79456e55-d5f6-4965-8790-d8190e9af42e"). InnerVolumeSpecName "kube-api-access-k5vhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:31:39.668263 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.668230 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79456e55-d5f6-4965-8790-d8190e9af42e-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:31:39.668263 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.668258 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5vhh\" (UniqueName: \"kubernetes.io/projected/79456e55-d5f6-4965-8790-d8190e9af42e-kube-api-access-k5vhh\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:31:39.668263 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:39.668268 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79456e55-d5f6-4965-8790-d8190e9af42e-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:31:40.036091 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:40.036058 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a82f1c3d-448f-4bef-86b6-b5bb68f83c5a" path="/var/lib/kubelet/pods/a82f1c3d-448f-4bef-86b6-b5bb68f83c5a/volumes" Jul 30 23:31:40.392913 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:40.392825 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" event={"ID":"79456e55-d5f6-4965-8790-d8190e9af42e","Type":"ContainerDied","Data":"11b1f11c05bf4c9583de646a8a42b1510cb5144aa2b70d85e6a5a66a1c37bd23"} Jul 30 23:31:40.392913 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:40.392889 2554 scope.go:117] "RemoveContainer" containerID="938b8c9338d5471a4f2c454be87a183298c0d7c4dca63fbf6ab4e42c0cc6546e" Jul 30 23:31:40.393431 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:40.392845 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84" Jul 30 23:31:40.409389 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:40.409363 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84"] Jul 30 23:31:40.415496 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:40.415463 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13c529-64988b5d97-c4c84"] Jul 30 23:31:42.036856 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:42.036825 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79456e55-d5f6-4965-8790-d8190e9af42e" path="/var/lib/kubelet/pods/79456e55-d5f6-4965-8790-d8190e9af42e/volumes" Jul 30 23:31:51.161076 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.161041 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49"] Jul 30 23:31:51.161745 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.161683 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79456e55-d5f6-4965-8790-d8190e9af42e" containerName="maas-api" Jul 30 23:31:51.161745 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.161706 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="79456e55-d5f6-4965-8790-d8190e9af42e" containerName="maas-api" Jul 30 23:31:51.161896 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.161749 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a82f1c3d-448f-4bef-86b6-b5bb68f83c5a" containerName="revoke-keys" Jul 30 23:31:51.161896 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.161759 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a82f1c3d-448f-4bef-86b6-b5bb68f83c5a" containerName="revoke-keys" Jul 30 23:31:51.162164 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.162142 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="79456e55-d5f6-4965-8790-d8190e9af42e" containerName="maas-api" Jul 30 23:31:51.162271 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.162216 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a82f1c3d-448f-4bef-86b6-b5bb68f83c5a" containerName="revoke-keys" Jul 30 23:31:51.165923 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.165901 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.168039 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.168013 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-5049e4-serving-cert\"" Jul 30 23:31:51.179523 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.179496 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49"] Jul 30 23:31:51.274679 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.274644 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.274860 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.274685 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.274860 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.274839 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7cmm\" (UniqueName: \"kubernetes.io/projected/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-kube-api-access-r7cmm\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.376121 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.376083 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.376121 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.376126 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.376367 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.376182 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7cmm\" (UniqueName: \"kubernetes.io/projected/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-kube-api-access-r7cmm\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.376367 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:31:51.376259 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-serving-cert: secret "maas-api-e2e-shared-a-5049e4-serving-cert" not found Jul 30 23:31:51.376367 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:31:51.376359 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls podName:3b3c38f7-2fc4-45ca-a87a-5225825b02f1 nodeName:}" failed. No retries permitted until 2026-07-30 23:31:51.876336282 +0000 UTC m=+2658.348800273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls") pod "maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" (UID: "3b3c38f7-2fc4-45ca-a87a-5225825b02f1") : secret "maas-api-e2e-shared-a-5049e4-serving-cert" not found Jul 30 23:31:51.376979 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.376957 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.394038 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.394003 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7cmm\" (UniqueName: \"kubernetes.io/projected/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-kube-api-access-r7cmm\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.882110 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.882068 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:51.884610 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:51.884582 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls\") pod \"maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:52.078428 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:52.078395 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:52.221037 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:52.221008 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49"] Jul 30 23:31:52.223076 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:31:52.223049 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3c38f7_2fc4_45ca_a87a_5225825b02f1.slice/crio-2ec02122ac09f017980433c581b89e8979a643b8c1cc2cddb8d16f74eb9ec965 WatchSource:0}: Error finding container 2ec02122ac09f017980433c581b89e8979a643b8c1cc2cddb8d16f74eb9ec965: Status 404 returned error can't find the container with id 2ec02122ac09f017980433c581b89e8979a643b8c1cc2cddb8d16f74eb9ec965 Jul 30 23:31:52.457648 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:52.457608 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" event={"ID":"3b3c38f7-2fc4-45ca-a87a-5225825b02f1","Type":"ContainerStarted","Data":"2ec02122ac09f017980433c581b89e8979a643b8c1cc2cddb8d16f74eb9ec965"} Jul 30 23:31:53.464759 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:53.464707 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" event={"ID":"3b3c38f7-2fc4-45ca-a87a-5225825b02f1","Type":"ContainerStarted","Data":"f7a511b5c3f481eadd4a4e932ec18f3c706cda020b48fe724885d7b4aa02d1a7"} Jul 30 23:31:53.465267 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:53.464843 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:31:53.483729 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:53.483680 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" podStartSLOduration=2.10932078 podStartE2EDuration="2.483664564s" totalImagesPullingTime="374.343784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:31:53.481131608 +0000 UTC m=+2659.953595619" watchObservedRunningTime="2026-07-30 23:31:53.483664564 +0000 UTC m=+2659.956128576" Jul 30 23:31:59.476314 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:31:59.476279 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:32:25.344820 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.344779 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f"] Jul 30 23:32:25.349162 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.349138 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:25.351895 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.351872 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f6f660-serving-cert\"" Jul 30 23:32:25.361352 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.361325 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f"] Jul 30 23:32:25.482629 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.482588 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:25.482861 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.482635 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:25.482861 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.482775 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8265w\" (UniqueName: \"kubernetes.io/projected/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-kube-api-access-8265w\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:25.583589 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.583552 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8265w\" (UniqueName: \"kubernetes.io/projected/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-kube-api-access-8265w\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:25.583808 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.583612 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:25.583808 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.583713 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:25.583945 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:32:25.583848 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-serving-cert: secret "maas-api-e2e-shared-b-f6f660-serving-cert" not found Jul 30 23:32:25.583945 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:32:25.583939 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls podName:b7ddd2a8-1430-489b-9a0e-a0023fb98a52 nodeName:}" failed. No retries permitted until 2026-07-30 23:32:26.083916528 +0000 UTC m=+2692.556380525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls") pod "maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" (UID: "b7ddd2a8-1430-489b-9a0e-a0023fb98a52") : secret "maas-api-e2e-shared-b-f6f660-serving-cert" not found Jul 30 23:32:25.584306 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.584288 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:25.595198 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:25.595140 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8265w\" (UniqueName: \"kubernetes.io/projected/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-kube-api-access-8265w\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:26.089118 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:26.089075 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:26.091778 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:26.091723 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls\") pod \"maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:26.262118 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:26.262085 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:26.405562 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:26.405528 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f"] Jul 30 23:32:26.409149 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:32:26.409122 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ddd2a8_1430_489b_9a0e_a0023fb98a52.slice/crio-eb2efda62b00dcbc0f572696bb11485f3358e7d5ab34f2af64c854e2cd494708 WatchSource:0}: Error finding container eb2efda62b00dcbc0f572696bb11485f3358e7d5ab34f2af64c854e2cd494708: Status 404 returned error can't find the container with id eb2efda62b00dcbc0f572696bb11485f3358e7d5ab34f2af64c854e2cd494708 Jul 30 23:32:26.644727 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:26.644634 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" event={"ID":"b7ddd2a8-1430-489b-9a0e-a0023fb98a52","Type":"ContainerStarted","Data":"eb2efda62b00dcbc0f572696bb11485f3358e7d5ab34f2af64c854e2cd494708"} Jul 30 23:32:27.651062 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:27.651028 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" event={"ID":"b7ddd2a8-1430-489b-9a0e-a0023fb98a52","Type":"ContainerStarted","Data":"1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0"} Jul 30 23:32:27.651481 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:27.651169 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:27.669703 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:27.669655 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" podStartSLOduration=2.26992472 podStartE2EDuration="2.669640816s" totalImagesPullingTime="399.716096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:32:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:32:27.667760288 +0000 UTC m=+2694.140224293" watchObservedRunningTime="2026-07-30 23:32:27.669640816 +0000 UTC m=+2694.142104828" Jul 30 23:32:33.662242 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:33.662213 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:32:34.186302 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:34.186277 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:32:34.207854 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:32:34.207830 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:35:34.361375 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:35:34.361303 2554 scope.go:117] "RemoveContainer" containerID="ecbd984afec25e45f6bd635d685fdf94cafe868b8f42399f6044245f8de2bb66" Jul 30 23:36:34.381499 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:36:34.381469 2554 scope.go:117] "RemoveContainer" containerID="3301339568fcc77c0e23c183fc926129346386c04fdc69a70ba32976627d0a03" Jul 30 23:37:34.226146 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:37:34.226121 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:37:34.239924 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:37:34.239902 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:37:34.402679 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:37:34.402649 2554 scope.go:117] "RemoveContainer" containerID="21fbb9884d2f8f3a699779c783f458666565dd7cb6b8d0ae0697c9925d102cc6" Jul 30 23:40:33.050418 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.050383 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z"] Jul 30 23:40:33.054384 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.054362 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.056825 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.056798 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-05d70d-serving-cert\"" Jul 30 23:40:33.066648 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.066621 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z"] Jul 30 23:40:33.167912 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.167880 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9fkg\" (UniqueName: \"kubernetes.io/projected/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-kube-api-access-n9fkg\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.168086 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.167943 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.168086 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.167961 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.269435 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.269388 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9fkg\" (UniqueName: \"kubernetes.io/projected/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-kube-api-access-n9fkg\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.269636 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.269452 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.269636 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.269473 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.269636 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:40:33.269616 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-serving-cert: secret "maas-api-e2e-ipp-a-05d70d-serving-cert" not found Jul 30 23:40:33.269848 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:40:33.269679 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls podName:a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f nodeName:}" failed. No retries permitted until 2026-07-30 23:40:33.769662285 +0000 UTC m=+3180.242126294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls") pod "maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" (UID: "a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f") : secret "maas-api-e2e-ipp-a-05d70d-serving-cert" not found Jul 30 23:40:33.270138 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.270120 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.278501 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.278478 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9fkg\" (UniqueName: \"kubernetes.io/projected/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-kube-api-access-n9fkg\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.773817 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.773771 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.776314 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.776284 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:33.968328 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:33.968295 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:34.111131 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:34.111102 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z"] Jul 30 23:40:34.113533 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:40:34.113499 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a5195d_a81b_4b97_9a4a_9bb4d1f8aa2f.slice/crio-32e13ae1f28d03434e45c6000b4d07037f1e7e9b50caafe5ade525550b8eb622 WatchSource:0}: Error finding container 32e13ae1f28d03434e45c6000b4d07037f1e7e9b50caafe5ade525550b8eb622: Status 404 returned error can't find the container with id 32e13ae1f28d03434e45c6000b4d07037f1e7e9b50caafe5ade525550b8eb622 Jul 30 23:40:34.114974 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:34.114953 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 23:40:34.170198 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:34.170164 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" event={"ID":"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f","Type":"ContainerStarted","Data":"32e13ae1f28d03434e45c6000b4d07037f1e7e9b50caafe5ade525550b8eb622"} Jul 30 23:40:35.177392 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:35.177339 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" event={"ID":"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f","Type":"ContainerStarted","Data":"38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445"} Jul 30 23:40:35.177866 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:35.177440 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:40:35.195890 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:35.195842 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" podStartSLOduration=1.785704699 podStartE2EDuration="2.195826921s" totalImagesPullingTime="410.122222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:40:35.193355897 +0000 UTC m=+3181.665819910" watchObservedRunningTime="2026-07-30 23:40:35.195826921 +0000 UTC m=+3181.668290932" Jul 30 23:40:41.188101 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:40:41.188065 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:41:04.463031 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.462992 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs"] Jul 30 23:41:04.467597 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.467571 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:04.470010 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.469983 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-25ff25-serving-cert\"" Jul 30 23:41:04.478518 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.478442 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs"] Jul 30 23:41:04.565852 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.565810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:04.566059 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.565947 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkcnx\" (UniqueName: \"kubernetes.io/projected/087ca4f2-f703-446e-be06-c5052cf73613-kube-api-access-tkcnx\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:04.566059 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.566012 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/087ca4f2-f703-446e-be06-c5052cf73613-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:04.667462 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.667424 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkcnx\" (UniqueName: \"kubernetes.io/projected/087ca4f2-f703-446e-be06-c5052cf73613-kube-api-access-tkcnx\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:04.667662 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.667493 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/087ca4f2-f703-446e-be06-c5052cf73613-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:04.667662 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.667531 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:04.667786 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:41:04.667681 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-serving-cert: secret "maas-api-e2e-ipp-b-25ff25-serving-cert" not found Jul 30 23:41:04.667843 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:41:04.667829 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls podName:087ca4f2-f703-446e-be06-c5052cf73613 nodeName:}" failed. No retries permitted until 2026-07-30 23:41:05.167795303 +0000 UTC m=+3211.640259300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls") pod "maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" (UID: "087ca4f2-f703-446e-be06-c5052cf73613") : secret "maas-api-e2e-ipp-b-25ff25-serving-cert" not found Jul 30 23:41:04.668253 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.668235 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/087ca4f2-f703-446e-be06-c5052cf73613-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:04.676282 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:04.676252 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkcnx\" (UniqueName: \"kubernetes.io/projected/087ca4f2-f703-446e-be06-c5052cf73613-kube-api-access-tkcnx\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:05.172655 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:05.172617 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:05.175196 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:05.175176 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls\") pod \"maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:05.383199 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:05.383156 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:05.529832 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:05.529797 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs"] Jul 30 23:41:05.531773 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:41:05.531717 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod087ca4f2_f703_446e_be06_c5052cf73613.slice/crio-4925ee44b647c8653e31cda557a878ccc4011c0e6def2bb6b0684494e6f32c49 WatchSource:0}: Error finding container 4925ee44b647c8653e31cda557a878ccc4011c0e6def2bb6b0684494e6f32c49: Status 404 returned error can't find the container with id 4925ee44b647c8653e31cda557a878ccc4011c0e6def2bb6b0684494e6f32c49 Jul 30 23:41:06.337558 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:06.337458 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" event={"ID":"087ca4f2-f703-446e-be06-c5052cf73613","Type":"ContainerStarted","Data":"317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137"} Jul 30 23:41:06.337558 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:06.337523 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" event={"ID":"087ca4f2-f703-446e-be06-c5052cf73613","Type":"ContainerStarted","Data":"4925ee44b647c8653e31cda557a878ccc4011c0e6def2bb6b0684494e6f32c49"} Jul 30 23:41:06.337813 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:06.337647 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:06.355966 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:06.355911 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" podStartSLOduration=1.944424946 podStartE2EDuration="2.35589524s" totalImagesPullingTime="411.470294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:41:06.353919659 +0000 UTC m=+3212.826383672" watchObservedRunningTime="2026-07-30 23:41:06.35589524 +0000 UTC m=+3212.828359283" Jul 30 23:41:12.347873 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:12.347844 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:41:35.212794 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:41:35.212762 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cq57g_4149d8bc-83ab-4403-bb3b-b43ba23413a0/payload-processing/0.log" Jul 30 23:42:25.241510 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:25.241481 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cq57g_4149d8bc-83ab-4403-bb3b-b43ba23413a0/payload-processing/0.log" Jul 30 23:42:29.979864 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:29.979830 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8"] Jul 30 23:42:29.983667 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:29.983642 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:29.986052 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:29.986019 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-07db9c-serving-cert\"" Jul 30 23:42:29.995393 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:29.994686 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8"] Jul 30 23:42:30.071997 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.071953 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cvpw\" (UniqueName: \"kubernetes.io/projected/f1559d39-8045-4abd-9a61-962de07b3870-kube-api-access-6cvpw\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.072180 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.072019 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1559d39-8045-4abd-9a61-962de07b3870-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.072180 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.072079 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.173267 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.173229 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cvpw\" (UniqueName: \"kubernetes.io/projected/f1559d39-8045-4abd-9a61-962de07b3870-kube-api-access-6cvpw\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.173449 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.173281 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1559d39-8045-4abd-9a61-962de07b3870-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.173449 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.173335 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.173528 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:42:30.173450 2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-serving-cert: secret "maas-api-e2e-ipp-cleanup-07db9c-serving-cert" not found Jul 30 23:42:30.173528 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:42:30.173501 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls podName:f1559d39-8045-4abd-9a61-962de07b3870 nodeName:}" failed. No retries permitted until 2026-07-30 23:42:30.673485261 +0000 UTC m=+3297.145949250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" (UID: "f1559d39-8045-4abd-9a61-962de07b3870") : secret "maas-api-e2e-ipp-cleanup-07db9c-serving-cert" not found Jul 30 23:42:30.174154 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.174128 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1559d39-8045-4abd-9a61-962de07b3870-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.185395 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.185365 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cvpw\" (UniqueName: \"kubernetes.io/projected/f1559d39-8045-4abd-9a61-962de07b3870-kube-api-access-6cvpw\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.678916 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.678882 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.681292 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.681266 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:30.898191 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:30.898154 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:31.047506 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:31.047442 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8"] Jul 30 23:42:31.050355 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:42:31.050326 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1559d39_8045_4abd_9a61_962de07b3870.slice/crio-cdedb3cb125d4379027f22a98a244d2dc1e7ab5558efcbb9ed61896003e37958 WatchSource:0}: Error finding container cdedb3cb125d4379027f22a98a244d2dc1e7ab5558efcbb9ed61896003e37958: Status 404 returned error can't find the container with id cdedb3cb125d4379027f22a98a244d2dc1e7ab5558efcbb9ed61896003e37958 Jul 30 23:42:31.774889 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:31.774793 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" event={"ID":"f1559d39-8045-4abd-9a61-962de07b3870","Type":"ContainerStarted","Data":"f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0"} Jul 30 23:42:31.774889 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:31.774846 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" event={"ID":"f1559d39-8045-4abd-9a61-962de07b3870","Type":"ContainerStarted","Data":"cdedb3cb125d4379027f22a98a244d2dc1e7ab5558efcbb9ed61896003e37958"} Jul 30 23:42:31.775142 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:31.774926 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:42:31.791680 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:31.791631 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" podStartSLOduration=2.416376733 podStartE2EDuration="2.791616609s" totalImagesPullingTime="375.239876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:42:31.789109291 +0000 UTC m=+3298.261573303" watchObservedRunningTime="2026-07-30 23:42:31.791616609 +0000 UTC m=+3298.264080600" Jul 30 23:42:34.257112 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:34.257081 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:42:34.272409 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:34.272386 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:42:37.786193 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:42:37.786163 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:43:00.597422 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.597387 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8"] Jul 30 23:43:00.598030 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.597632 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" podUID="f1559d39-8045-4abd-9a61-962de07b3870" containerName="maas-api" containerID="cri-o://f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0" gracePeriod=30 Jul 30 23:43:00.865306 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.865280 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:43:00.938944 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.938904 2554 generic.go:356] "Generic (PLEG): container finished" podID="f1559d39-8045-4abd-9a61-962de07b3870" containerID="f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0" exitCode=0 Jul 30 23:43:00.939146 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.938954 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" Jul 30 23:43:00.939146 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.938992 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" event={"ID":"f1559d39-8045-4abd-9a61-962de07b3870","Type":"ContainerDied","Data":"f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0"} Jul 30 23:43:00.939146 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.939056 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8" event={"ID":"f1559d39-8045-4abd-9a61-962de07b3870","Type":"ContainerDied","Data":"cdedb3cb125d4379027f22a98a244d2dc1e7ab5558efcbb9ed61896003e37958"} Jul 30 23:43:00.939146 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.939087 2554 scope.go:117] "RemoveContainer" containerID="f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0" Jul 30 23:43:00.947937 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.947916 2554 scope.go:117] "RemoveContainer" containerID="f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0" Jul 30 23:43:00.948234 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:43:00.948208 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0\": container with ID starting with f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0 not found: ID does not exist" containerID="f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0" Jul 30 23:43:00.948328 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.948242 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0"} err="failed to get container status \"f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0\": rpc error: code = NotFound desc = could not find container \"f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0\": container with ID starting with f37c71d9581e1631b6ae4ed45f2067ff440d71a0e5cd67f940e48d359b4e9bd0 not found: ID does not exist" Jul 30 23:43:00.971785 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.971755 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cvpw\" (UniqueName: \"kubernetes.io/projected/f1559d39-8045-4abd-9a61-962de07b3870-kube-api-access-6cvpw\") pod \"f1559d39-8045-4abd-9a61-962de07b3870\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " Jul 30 23:43:00.971963 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.971801 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls\") pod \"f1559d39-8045-4abd-9a61-962de07b3870\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " Jul 30 23:43:00.971963 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.971877 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1559d39-8045-4abd-9a61-962de07b3870-openshift-service-ca\") pod \"f1559d39-8045-4abd-9a61-962de07b3870\" (UID: \"f1559d39-8045-4abd-9a61-962de07b3870\") " Jul 30 23:43:00.972212 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.972181 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1559d39-8045-4abd-9a61-962de07b3870-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1559d39-8045-4abd-9a61-962de07b3870" (UID: "f1559d39-8045-4abd-9a61-962de07b3870"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:43:00.973990 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.973961 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1559d39-8045-4abd-9a61-962de07b3870-kube-api-access-6cvpw" (OuterVolumeSpecName: "kube-api-access-6cvpw") pod "f1559d39-8045-4abd-9a61-962de07b3870" (UID: "f1559d39-8045-4abd-9a61-962de07b3870"). InnerVolumeSpecName "kube-api-access-6cvpw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:43:00.974093 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:00.974042 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1559d39-8045-4abd-9a61-962de07b3870" (UID: "f1559d39-8045-4abd-9a61-962de07b3870"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:43:01.072875 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:01.072833 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1559d39-8045-4abd-9a61-962de07b3870-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:43:01.072875 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:01.072869 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cvpw\" (UniqueName: \"kubernetes.io/projected/f1559d39-8045-4abd-9a61-962de07b3870-kube-api-access-6cvpw\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:43:01.072875 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:01.072880 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1559d39-8045-4abd-9a61-962de07b3870-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:43:01.261522 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:01.261469 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8"] Jul 30 23:43:01.263691 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:01.263663 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07db9c-fc7549dd8-ph8b8"] Jul 30 23:43:02.036364 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:02.036329 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1559d39-8045-4abd-9a61-962de07b3870" path="/var/lib/kubelet/pods/f1559d39-8045-4abd-9a61-962de07b3870/volumes" Jul 30 23:43:26.377095 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.377059 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z"] Jul 30 23:43:26.377530 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.377269 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" podUID="a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" containerName="maas-api" containerID="cri-o://38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445" gracePeriod=30 Jul 30 23:43:26.653218 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.653190 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:43:26.707164 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.707134 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9fkg\" (UniqueName: \"kubernetes.io/projected/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-kube-api-access-n9fkg\") pod \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " Jul 30 23:43:26.707164 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.707169 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-openshift-service-ca\") pod \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " Jul 30 23:43:26.707422 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.707276 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls\") pod \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\" (UID: \"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f\") " Jul 30 23:43:26.707583 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.707555 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" (UID: "a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:43:26.709301 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.709277 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" (UID: "a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:43:26.709417 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.709327 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-kube-api-access-n9fkg" (OuterVolumeSpecName: "kube-api-access-n9fkg") pod "a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" (UID: "a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f"). InnerVolumeSpecName "kube-api-access-n9fkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:43:26.808753 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.808699 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9fkg\" (UniqueName: \"kubernetes.io/projected/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-kube-api-access-n9fkg\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:43:26.808753 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.808727 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:43:26.808949 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:26.808769 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:43:27.073528 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.073492 2554 generic.go:356] "Generic (PLEG): container finished" podID="a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" containerID="38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445" exitCode=0 Jul 30 23:43:27.073697 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.073538 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" Jul 30 23:43:27.073697 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.073535 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" event={"ID":"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f","Type":"ContainerDied","Data":"38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445"} Jul 30 23:43:27.073697 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.073667 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z" event={"ID":"a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f","Type":"ContainerDied","Data":"32e13ae1f28d03434e45c6000b4d07037f1e7e9b50caafe5ade525550b8eb622"} Jul 30 23:43:27.073829 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.073697 2554 scope.go:117] "RemoveContainer" containerID="38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445" Jul 30 23:43:27.083051 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.083033 2554 scope.go:117] "RemoveContainer" containerID="38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445" Jul 30 23:43:27.083285 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:43:27.083268 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445\": container with ID starting with 38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445 not found: ID does not exist" containerID="38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445" Jul 30 23:43:27.083338 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.083292 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445"} err="failed to get container status \"38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445\": rpc error: code = NotFound desc = could not find container \"38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445\": container with ID starting with 38d3249b8ee6dcae010f0ebb7080b08f18256d512749c5df3f39a96033c8c445 not found: ID does not exist" Jul 30 23:43:27.095153 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.095124 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z"] Jul 30 23:43:27.098251 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:27.098228 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-05d70d-5866c4756d-wzn2z"] Jul 30 23:43:28.037597 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:43:28.037548 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" path="/var/lib/kubelet/pods/a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f/volumes" Jul 30 23:44:05.859154 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:05.858512 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs"] Jul 30 23:44:05.859154 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:05.858807 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" podUID="087ca4f2-f703-446e-be06-c5052cf73613" containerName="maas-api" containerID="cri-o://317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137" gracePeriod=30 Jul 30 23:44:06.127000 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.126973 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:44:06.173036 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.173005 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkcnx\" (UniqueName: \"kubernetes.io/projected/087ca4f2-f703-446e-be06-c5052cf73613-kube-api-access-tkcnx\") pod \"087ca4f2-f703-446e-be06-c5052cf73613\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " Jul 30 23:44:06.173234 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.173112 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/087ca4f2-f703-446e-be06-c5052cf73613-openshift-service-ca\") pod \"087ca4f2-f703-446e-be06-c5052cf73613\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " Jul 30 23:44:06.173234 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.173156 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls\") pod \"087ca4f2-f703-446e-be06-c5052cf73613\" (UID: \"087ca4f2-f703-446e-be06-c5052cf73613\") " Jul 30 23:44:06.173479 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.173458 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/087ca4f2-f703-446e-be06-c5052cf73613-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "087ca4f2-f703-446e-be06-c5052cf73613" (UID: "087ca4f2-f703-446e-be06-c5052cf73613"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:44:06.175272 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.175234 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/087ca4f2-f703-446e-be06-c5052cf73613-kube-api-access-tkcnx" (OuterVolumeSpecName: "kube-api-access-tkcnx") pod "087ca4f2-f703-446e-be06-c5052cf73613" (UID: "087ca4f2-f703-446e-be06-c5052cf73613"). InnerVolumeSpecName "kube-api-access-tkcnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:44:06.175272 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.175242 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "087ca4f2-f703-446e-be06-c5052cf73613" (UID: "087ca4f2-f703-446e-be06-c5052cf73613"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:44:06.274699 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.274670 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/087ca4f2-f703-446e-be06-c5052cf73613-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:06.274699 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.274699 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/087ca4f2-f703-446e-be06-c5052cf73613-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:06.274996 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.274717 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkcnx\" (UniqueName: \"kubernetes.io/projected/087ca4f2-f703-446e-be06-c5052cf73613-kube-api-access-tkcnx\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:06.276064 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.276030 2554 generic.go:356] "Generic (PLEG): container finished" podID="087ca4f2-f703-446e-be06-c5052cf73613" containerID="317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137" exitCode=0 Jul 30 23:44:06.276189 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.276070 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" Jul 30 23:44:06.276189 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.276110 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" event={"ID":"087ca4f2-f703-446e-be06-c5052cf73613","Type":"ContainerDied","Data":"317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137"} Jul 30 23:44:06.276189 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.276176 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs" event={"ID":"087ca4f2-f703-446e-be06-c5052cf73613","Type":"ContainerDied","Data":"4925ee44b647c8653e31cda557a878ccc4011c0e6def2bb6b0684494e6f32c49"} Jul 30 23:44:06.276304 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.276209 2554 scope.go:117] "RemoveContainer" containerID="317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137" Jul 30 23:44:06.286596 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.286571 2554 scope.go:117] "RemoveContainer" containerID="317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137" Jul 30 23:44:06.286946 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:44:06.286924 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137\": container with ID starting with 317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137 not found: ID does not exist" containerID="317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137" Jul 30 23:44:06.287061 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.286953 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137"} err="failed to get container status \"317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137\": rpc error: code = NotFound desc = could not find container \"317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137\": container with ID starting with 317c5d3ae8f6a054a8e69cfe38b9b0df0f0dc3e3e065ecbfbfc762fc23637137 not found: ID does not exist" Jul 30 23:44:06.301472 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.301447 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs"] Jul 30 23:44:06.304946 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:06.304922 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-25ff25-d556fdfd9-q2scs"] Jul 30 23:44:08.037000 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:08.036967 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="087ca4f2-f703-446e-be06-c5052cf73613" path="/var/lib/kubelet/pods/087ca4f2-f703-446e-be06-c5052cf73613/volumes" Jul 30 23:44:23.132264 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.132023 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49"] Jul 30 23:44:23.132834 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.132268 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" podUID="3b3c38f7-2fc4-45ca-a87a-5225825b02f1" containerName="maas-api" containerID="cri-o://f7a511b5c3f481eadd4a4e932ec18f3c706cda020b48fe724885d7b4aa02d1a7" gracePeriod=30 Jul 30 23:44:23.366529 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.366496 2554 generic.go:356] "Generic (PLEG): container finished" podID="3b3c38f7-2fc4-45ca-a87a-5225825b02f1" containerID="f7a511b5c3f481eadd4a4e932ec18f3c706cda020b48fe724885d7b4aa02d1a7" exitCode=0 Jul 30 23:44:23.366702 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.366569 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" event={"ID":"3b3c38f7-2fc4-45ca-a87a-5225825b02f1","Type":"ContainerDied","Data":"f7a511b5c3f481eadd4a4e932ec18f3c706cda020b48fe724885d7b4aa02d1a7"} Jul 30 23:44:23.388975 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.388912 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:44:23.431623 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.431592 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls\") pod \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " Jul 30 23:44:23.431815 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.431671 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-openshift-service-ca\") pod \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " Jul 30 23:44:23.431815 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.431792 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7cmm\" (UniqueName: \"kubernetes.io/projected/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-kube-api-access-r7cmm\") pod \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\" (UID: \"3b3c38f7-2fc4-45ca-a87a-5225825b02f1\") " Jul 30 23:44:23.432102 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.432075 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b3c38f7-2fc4-45ca-a87a-5225825b02f1" (UID: "3b3c38f7-2fc4-45ca-a87a-5225825b02f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:44:23.433713 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.433681 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b3c38f7-2fc4-45ca-a87a-5225825b02f1" (UID: "3b3c38f7-2fc4-45ca-a87a-5225825b02f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:44:23.433854 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.433783 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-kube-api-access-r7cmm" (OuterVolumeSpecName: "kube-api-access-r7cmm") pod "3b3c38f7-2fc4-45ca-a87a-5225825b02f1" (UID: "3b3c38f7-2fc4-45ca-a87a-5225825b02f1"). InnerVolumeSpecName "kube-api-access-r7cmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:44:23.532900 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.532871 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:23.532900 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.532896 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7cmm\" (UniqueName: \"kubernetes.io/projected/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-kube-api-access-r7cmm\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:23.533092 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:23.532908 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3c38f7-2fc4-45ca-a87a-5225825b02f1-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:24.372717 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:24.372640 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" Jul 30 23:44:24.373159 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:24.372638 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49" event={"ID":"3b3c38f7-2fc4-45ca-a87a-5225825b02f1","Type":"ContainerDied","Data":"2ec02122ac09f017980433c581b89e8979a643b8c1cc2cddb8d16f74eb9ec965"} Jul 30 23:44:24.373159 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:24.372786 2554 scope.go:117] "RemoveContainer" containerID="f7a511b5c3f481eadd4a4e932ec18f3c706cda020b48fe724885d7b4aa02d1a7" Jul 30 23:44:24.389054 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:24.389027 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49"] Jul 30 23:44:24.392629 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:24.392607 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5049e4-6b86c9d879-xzf49"] Jul 30 23:44:26.037082 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:26.037040 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b3c38f7-2fc4-45ca-a87a-5225825b02f1" path="/var/lib/kubelet/pods/3b3c38f7-2fc4-45ca-a87a-5225825b02f1/volumes" Jul 30 23:44:40.610561 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:40.610522 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f"] Jul 30 23:44:40.611212 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:40.610821 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" podUID="b7ddd2a8-1430-489b-9a0e-a0023fb98a52" containerName="maas-api" containerID="cri-o://1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0" gracePeriod=30 Jul 30 23:44:40.872910 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:40.872849 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:44:41.000449 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.000415 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls\") pod \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " Jul 30 23:44:41.000642 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.000503 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-openshift-service-ca\") pod \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " Jul 30 23:44:41.000642 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.000538 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8265w\" (UniqueName: \"kubernetes.io/projected/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-kube-api-access-8265w\") pod \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\" (UID: \"b7ddd2a8-1430-489b-9a0e-a0023fb98a52\") " Jul 30 23:44:41.000977 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.000941 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7ddd2a8-1430-489b-9a0e-a0023fb98a52" (UID: "b7ddd2a8-1430-489b-9a0e-a0023fb98a52"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 23:44:41.002638 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.002609 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7ddd2a8-1430-489b-9a0e-a0023fb98a52" (UID: "b7ddd2a8-1430-489b-9a0e-a0023fb98a52"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 23:44:41.002832 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.002813 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-kube-api-access-8265w" (OuterVolumeSpecName: "kube-api-access-8265w") pod "b7ddd2a8-1430-489b-9a0e-a0023fb98a52" (UID: "b7ddd2a8-1430-489b-9a0e-a0023fb98a52"). InnerVolumeSpecName "kube-api-access-8265w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 23:44:41.101563 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.101531 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-maas-api-tls\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:41.101563 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.101563 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-openshift-service-ca\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:41.101795 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.101573 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8265w\" (UniqueName: \"kubernetes.io/projected/b7ddd2a8-1430-489b-9a0e-a0023fb98a52-kube-api-access-8265w\") on node \"ip-10-0-131-130.ec2.internal\" DevicePath \"\"" Jul 30 23:44:41.461914 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.461879 2554 generic.go:356] "Generic (PLEG): container finished" podID="b7ddd2a8-1430-489b-9a0e-a0023fb98a52" containerID="1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0" exitCode=0 Jul 30 23:44:41.462114 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.461926 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" Jul 30 23:44:41.462114 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.461963 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" event={"ID":"b7ddd2a8-1430-489b-9a0e-a0023fb98a52","Type":"ContainerDied","Data":"1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0"} Jul 30 23:44:41.462114 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.462012 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f" event={"ID":"b7ddd2a8-1430-489b-9a0e-a0023fb98a52","Type":"ContainerDied","Data":"eb2efda62b00dcbc0f572696bb11485f3358e7d5ab34f2af64c854e2cd494708"} Jul 30 23:44:41.462114 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.462034 2554 scope.go:117] "RemoveContainer" containerID="1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0" Jul 30 23:44:41.471192 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.471175 2554 scope.go:117] "RemoveContainer" containerID="1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0" Jul 30 23:44:41.471470 ip-10-0-131-130 kubenswrapper[2554]: E0730 23:44:41.471451 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0\": container with ID starting with 1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0 not found: ID does not exist" containerID="1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0" Jul 30 23:44:41.471514 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.471477 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0"} err="failed to get container status \"1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0\": rpc error: code = NotFound desc = could not find container \"1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0\": container with ID starting with 1f28ea304ae766d4507baaaed50d509d8c4ba1c16d52efde2fcfb2277cc1a8d0 not found: ID does not exist" Jul 30 23:44:41.483780 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.483729 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f"] Jul 30 23:44:41.488107 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:41.488082 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f6f660-78c7d75d9c-s5w4f"] Jul 30 23:44:42.036644 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:42.036612 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7ddd2a8-1430-489b-9a0e-a0023fb98a52" path="/var/lib/kubelet/pods/b7ddd2a8-1430-489b-9a0e-a0023fb98a52/volumes" Jul 30 23:44:53.644540 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:53.644508 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w9ktl_8feacd87-042b-4901-a834-81638db155c3/manager/0.log" Jul 30 23:44:53.760428 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:53.760394 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6hmd6_22647def-c7dc-43d6-9329-699e1980f14b/manager/0.log" Jul 30 23:44:53.878062 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:53.878031 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66ff69c46-pz992_b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254/manager/0.log" Jul 30 23:44:54.108516 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:54.108480 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lz69s_cc1d56d4-e784-44ad-af2e-a6865ff55437/manager/0.log" Jul 30 23:44:54.336553 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:54.336526 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c6bb66584-dgr2p_78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61/copy-manifests/0.log" Jul 30 23:44:54.348380 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:54.348350 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c6bb66584-dgr2p_78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61/manager/0.log" Jul 30 23:44:54.806587 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:54.806561 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7dcf74846d-h78k8_fa073fc7-6d1d-4da7-b0c9-0ae5c6272fb3/maas-api/0.log" Jul 30 23:44:55.243136 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:55.243108 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6tncd_a7ea2c6d-d14c-40d7-99d7-99964f54211a/postgres/0.log" Jul 30 23:44:56.133167 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:56.133136 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-m8f5q_6954eb3a-cfdb-4366-95d5-8e88dacb394d/manager/0.log" Jul 30 23:44:56.243659 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:56.243634 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-rkf9j_71f3036e-61ae-4e9f-8d35-3a274b69d054/manager/0.log" Jul 30 23:44:56.355095 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:56.355070 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-m9x2m_37a2509b-a9d3-42eb-8ca4-8388ce6dc247/kuadrant-console-plugin/0.log" Jul 30 23:44:56.690866 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:56.690831 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-mnk6w_bfe06b27-7e9a-49ae-b271-abdd97d57d22/manager/0.log" Jul 30 23:44:57.255123 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:57.255094 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-556f7c5984-hk98v_3ab46eeb-d12d-451c-94a1-b5bd521e5680/kube-auth-proxy/0.log" Jul 30 23:44:57.586566 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:57.586491 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-h9b6s_0d7648de-0da2-43be-abf6-d74241ca1f9b/payload-pre-processing/0.log" Jul 30 23:44:57.700271 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:57.700241 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cq57g_4149d8bc-83ab-4403-bb3b-b43ba23413a0/payload-processing/0.log" Jul 30 23:44:57.807778 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:57.807754 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66f84c8f94-gds4x_d8393df3-ae76-4d13-bbc1-1bfe1305cb87/router/0.log" Jul 30 23:44:58.156362 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:58.156335 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xmsdv_a067d259-83ae-42b2-ab12-caf6124c104c/main/0.log" Jul 30 23:44:58.269854 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:58.269820 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-m6ncc_143b975a-e272-4624-badf-de435e717efb/main/0.log" Jul 30 23:44:58.609799 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:44:58.609770 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-jhhks_5c4acb68-8444-41d8-8805-8651a7cc550a/main/0.log" Jul 30 23:45:06.709452 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:06.709410 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-spp8w_ff6d597e-b9e7-4074-b08e-0ab2234d4a3a/global-pull-secret-syncer/0.log" Jul 30 23:45:06.831355 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:06.831323 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t9f9p_2ab13381-4217-4fbd-a559-07f05259738f/konnectivity-agent/0.log" Jul 30 23:45:06.852548 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:06.852523 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-130.ec2.internal_15e562856c5ed1d4d691baee256e1db7/haproxy/0.log" Jul 30 23:45:12.056331 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:12.056236 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-m8f5q_6954eb3a-cfdb-4366-95d5-8e88dacb394d/manager/0.log" Jul 30 23:45:12.083213 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:12.083188 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-rkf9j_71f3036e-61ae-4e9f-8d35-3a274b69d054/manager/0.log" Jul 30 23:45:12.109501 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:12.109466 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-m9x2m_37a2509b-a9d3-42eb-8ca4-8388ce6dc247/kuadrant-console-plugin/0.log" Jul 30 23:45:12.321047 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:12.320957 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-mnk6w_bfe06b27-7e9a-49ae-b271-abdd97d57d22/manager/0.log" Jul 30 23:45:14.028170 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:14.028141 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-vh474_90fd9a18-a33a-45c5-b685-20e709a56cf7/cluster-monitoring-operator/0.log" Jul 30 23:45:14.191488 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:14.191448 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cvzk6_34fba92e-685c-43d5-8459-a0e0fd7cfef9/node-exporter/0.log" Jul 30 23:45:14.212399 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:14.212377 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cvzk6_34fba92e-685c-43d5-8459-a0e0fd7cfef9/kube-rbac-proxy/0.log" Jul 30 23:45:14.235771 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:14.235723 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cvzk6_34fba92e-685c-43d5-8459-a0e0fd7cfef9/init-textfile/0.log" Jul 30 23:45:16.479926 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:16.479884 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-zjbk6_2bd51092-0adc-481e-bc30-83d4604f2513/networking-console-plugin/0.log" Jul 30 23:45:17.088702 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.088675 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/1.log" Jul 30 23:45:17.094016 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.093992 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bfq27_5ba57d39-8636-40b7-b529-906527b1c207/console-operator/2.log" Jul 30 23:45:17.531509 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.531474 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb"] Jul 30 23:45:17.531974 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.531951 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" containerName="maas-api" Jul 30 23:45:17.531974 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.531972 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.531992 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b3c38f7-2fc4-45ca-a87a-5225825b02f1" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532000 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b3c38f7-2fc4-45ca-a87a-5225825b02f1" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532009 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="087ca4f2-f703-446e-be06-c5052cf73613" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532019 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="087ca4f2-f703-446e-be06-c5052cf73613" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532042 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1559d39-8045-4abd-9a61-962de07b3870" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532050 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1559d39-8045-4abd-9a61-962de07b3870" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532060 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7ddd2a8-1430-489b-9a0e-a0023fb98a52" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532070 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7ddd2a8-1430-489b-9a0e-a0023fb98a52" containerName="maas-api" Jul 30 23:45:17.532183 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532186 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a5195d-a81b-4b97-9a4a-9bb4d1f8aa2f" containerName="maas-api" Jul 30 23:45:17.532469 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532202 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7ddd2a8-1430-489b-9a0e-a0023fb98a52" containerName="maas-api" Jul 30 23:45:17.532469 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532211 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b3c38f7-2fc4-45ca-a87a-5225825b02f1" containerName="maas-api" Jul 30 23:45:17.532469 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532217 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="087ca4f2-f703-446e-be06-c5052cf73613" containerName="maas-api" Jul 30 23:45:17.532469 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.532223 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1559d39-8045-4abd-9a61-962de07b3870" containerName="maas-api" Jul 30 23:45:17.535704 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.535686 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.538328 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.538308 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b87wl\"/\"kube-root-ca.crt\"" Jul 30 23:45:17.538448 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.538371 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b87wl\"/\"default-dockercfg-m4wrr\"" Jul 30 23:45:17.538448 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.538404 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b87wl\"/\"openshift-service-ca.crt\"" Jul 30 23:45:17.544514 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.544490 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb"] Jul 30 23:45:17.607877 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.607844 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-sys\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.607877 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.607880 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-proc\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.608141 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.607903 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vtrl\" (UniqueName: \"kubernetes.io/projected/d195e5a8-c606-46f7-9b84-d5674767fbef-kube-api-access-6vtrl\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.608141 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.608026 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-podres\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.608141 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.608069 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-lib-modules\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.708788 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.708756 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-sys\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.708982 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.708801 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-proc\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.708982 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.708835 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vtrl\" (UniqueName: \"kubernetes.io/projected/d195e5a8-c606-46f7-9b84-d5674767fbef-kube-api-access-6vtrl\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.708982 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.708865 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-sys\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.708982 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.708906 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-podres\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.708982 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.708919 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-proc\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.708982 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.708949 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-lib-modules\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.709261 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.709016 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-podres\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.709261 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.709111 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d195e5a8-c606-46f7-9b84-d5674767fbef-lib-modules\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.717080 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.717053 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vtrl\" (UniqueName: \"kubernetes.io/projected/d195e5a8-c606-46f7-9b84-d5674767fbef-kube-api-access-6vtrl\") pod \"perf-node-gather-daemonset-crkrb\" (UID: \"d195e5a8-c606-46f7-9b84-d5674767fbef\") " pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.764856 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.764826 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-h9m6j_5095bdb3-4d29-46e0-8fe8-9f351e438b01/download-server/0.log" Jul 30 23:45:17.846422 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.846329 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:17.982012 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:17.981982 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb"] Jul 30 23:45:17.983922 ip-10-0-131-130 kubenswrapper[2554]: W0730 23:45:17.983890 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd195e5a8_c606_46f7_9b84_d5674767fbef.slice/crio-5c3914427476cac603097f4b8e28a4718945af3c84a54be6eab6b5880f5da4af WatchSource:0}: Error finding container 5c3914427476cac603097f4b8e28a4718945af3c84a54be6eab6b5880f5da4af: Status 404 returned error can't find the container with id 5c3914427476cac603097f4b8e28a4718945af3c84a54be6eab6b5880f5da4af Jul 30 23:45:18.493491 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:18.493464 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-bx5lq_93654742-1acd-4b8c-97ca-1cf1d5c73570/volume-data-source-validator/0.log" Jul 30 23:45:18.652067 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:18.652034 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" event={"ID":"d195e5a8-c606-46f7-9b84-d5674767fbef","Type":"ContainerStarted","Data":"e671407870370250a409cc29007866199823b00d0613b677eb8a4dc6f393a714"} Jul 30 23:45:18.652468 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:18.652092 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" event={"ID":"d195e5a8-c606-46f7-9b84-d5674767fbef","Type":"ContainerStarted","Data":"5c3914427476cac603097f4b8e28a4718945af3c84a54be6eab6b5880f5da4af"} Jul 30 23:45:18.652468 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:18.652230 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:18.673045 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:18.672997 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" podStartSLOduration=1.6729822269999999 podStartE2EDuration="1.672982227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 23:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:45:18.67205665 +0000 UTC m=+3465.144520662" watchObservedRunningTime="2026-07-30 23:45:18.672982227 +0000 UTC m=+3465.145446238" Jul 30 23:45:19.672064 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:19.672024 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fg66p_4bef5592-9ae3-4e19-9d0e-b309bceb1f59/dns/0.log" Jul 30 23:45:19.693834 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:19.693797 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fg66p_4bef5592-9ae3-4e19-9d0e-b309bceb1f59/kube-rbac-proxy/0.log" Jul 30 23:45:19.739154 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:19.739128 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mm57w_bf94eccb-32e3-45b3-9329-8991e4f7d720/dns-node-resolver/0.log" Jul 30 23:45:20.317826 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:20.317797 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9srwj_f340f7c6-8994-43b9-a22a-8b7a4883518c/node-ca/0.log" Jul 30 23:45:21.478249 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:21.478221 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-556f7c5984-hk98v_3ab46eeb-d12d-451c-94a1-b5bd521e5680/kube-auth-proxy/0.log" Jul 30 23:45:21.567107 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:21.567074 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-h9b6s_0d7648de-0da2-43be-abf6-d74241ca1f9b/payload-pre-processing/0.log" Jul 30 23:45:21.598856 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:21.598824 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cq57g_4149d8bc-83ab-4403-bb3b-b43ba23413a0/payload-processing/0.log" Jul 30 23:45:21.619339 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:21.619311 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66f84c8f94-gds4x_d8393df3-ae76-4d13-bbc1-1bfe1305cb87/router/0.log" Jul 30 23:45:22.449353 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:22.449325 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d8qbt_1d8434f7-e903-4268-aae4-fe78c1ab339e/serve-healthcheck-canary/0.log" Jul 30 23:45:22.939921 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:22.939887 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-bzz2p_fce2a95b-b468-4a4d-bf2f-76e040945724/insights-operator/0.log" Jul 30 23:45:22.942039 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:22.942019 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-bzz2p_fce2a95b-b468-4a4d-bf2f-76e040945724/insights-operator/1.log" Jul 30 23:45:22.963929 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:22.963898 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6gnrz_67d05f84-9319-4f44-8a23-57049b3834ac/kube-rbac-proxy/0.log" Jul 30 23:45:22.984121 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:22.984101 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6gnrz_67d05f84-9319-4f44-8a23-57049b3834ac/exporter/0.log" Jul 30 23:45:23.005301 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:23.005277 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6gnrz_67d05f84-9319-4f44-8a23-57049b3834ac/extractor/0.log" Jul 30 23:45:24.667442 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:24.667416 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b87wl/perf-node-gather-daemonset-crkrb" Jul 30 23:45:25.298308 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:25.298250 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w9ktl_8feacd87-042b-4901-a834-81638db155c3/manager/0.log" Jul 30 23:45:25.524552 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:25.524518 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6hmd6_22647def-c7dc-43d6-9329-699e1980f14b/manager/0.log" Jul 30 23:45:25.673440 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:25.673409 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66ff69c46-pz992_b35a9fba-bcc5-4bc6-a9bd-6d29d3cbc254/manager/0.log" Jul 30 23:45:25.763900 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:25.763866 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lz69s_cc1d56d4-e784-44ad-af2e-a6865ff55437/manager/0.log" Jul 30 23:45:25.900997 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:25.900966 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c6bb66584-dgr2p_78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61/manager/0.log" Jul 30 23:45:25.918353 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:25.918327 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c6bb66584-dgr2p_78a52fdd-4a4e-43cf-bbeb-d43fd5b2ea61/copy-manifests/0.log" Jul 30 23:45:28.352572 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:28.352538 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-556f7c5984-hk98v_3ab46eeb-d12d-451c-94a1-b5bd521e5680/kube-auth-proxy/0.log" Jul 30 23:45:28.413490 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:28.413455 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-h9b6s_0d7648de-0da2-43be-abf6-d74241ca1f9b/payload-pre-processing/0.log" Jul 30 23:45:28.432719 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:28.432688 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cq57g_4149d8bc-83ab-4403-bb3b-b43ba23413a0/payload-processing/0.log" Jul 30 23:45:28.447057 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:28.447027 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66f84c8f94-gds4x_d8393df3-ae76-4d13-bbc1-1bfe1305cb87/router/0.log" Jul 30 23:45:30.076369 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:30.076338 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-556f7c5984-hk98v_3ab46eeb-d12d-451c-94a1-b5bd521e5680/kube-auth-proxy/0.log" Jul 30 23:45:30.143652 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:30.143615 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-h9b6s_0d7648de-0da2-43be-abf6-d74241ca1f9b/payload-pre-processing/0.log" Jul 30 23:45:30.168717 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:30.168692 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cq57g_4149d8bc-83ab-4403-bb3b-b43ba23413a0/payload-processing/0.log" Jul 30 23:45:30.186491 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:30.186467 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66f84c8f94-gds4x_d8393df3-ae76-4d13-bbc1-1bfe1305cb87/router/0.log" Jul 30 23:45:32.450676 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:32.450639 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-5lpj4_e9818882-6d2d-4e4d-935c-8e0de57550ec/migrator/0.log" Jul 30 23:45:32.471121 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:32.471099 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-5lpj4_e9818882-6d2d-4e4d-935c-8e0de57550ec/graceful-termination/0.log" Jul 30 23:45:32.844560 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:32.844454 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-bcnkp_05000c78-03e9-4364-86c2-5aa37dd2dd14/kube-storage-version-migrator-operator/1.log" Jul 30 23:45:32.845250 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:32.845215 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-bcnkp_05000c78-03e9-4364-86c2-5aa37dd2dd14/kube-storage-version-migrator-operator/0.log" Jul 30 23:45:33.836205 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:33.836175 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfnwj_9ff92d75-8091-4a72-a19b-6a39fef8f5d9/kube-multus-additional-cni-plugins/0.log" Jul 30 23:45:33.859911 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:33.859890 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfnwj_9ff92d75-8091-4a72-a19b-6a39fef8f5d9/egress-router-binary-copy/0.log" Jul 30 23:45:33.882700 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:33.882676 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfnwj_9ff92d75-8091-4a72-a19b-6a39fef8f5d9/cni-plugins/0.log" Jul 30 23:45:33.906663 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:33.906636 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfnwj_9ff92d75-8091-4a72-a19b-6a39fef8f5d9/bond-cni-plugin/0.log" Jul 30 23:45:33.935472 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:33.935446 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfnwj_9ff92d75-8091-4a72-a19b-6a39fef8f5d9/routeoverride-cni/0.log" Jul 30 23:45:33.957665 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:33.957640 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfnwj_9ff92d75-8091-4a72-a19b-6a39fef8f5d9/whereabouts-cni-bincopy/0.log" Jul 30 23:45:33.978610 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:33.978587 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfnwj_9ff92d75-8091-4a72-a19b-6a39fef8f5d9/whereabouts-cni/0.log" Jul 30 23:45:34.325909 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:34.325879 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jqkvm_dfd556d1-cf73-4e72-9959-988c7d67484e/kube-multus/0.log" Jul 30 23:45:34.425590 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:34.425559 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2n7px_25f75d33-3b48-4851-b5d8-5e7cd1db81ed/network-metrics-daemon/0.log" Jul 30 23:45:34.448464 ip-10-0-131-130 kubenswrapper[2554]: I0730 23:45:34.448436 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2n7px_25f75d33-3b48-4851-b5d8-5e7cd1db81ed/kube-rbac-proxy/0.log"