Jul 06 14:24:24.334091 ip-10-0-128-205 systemd[1]: Starting Kubernetes Kubelet... Jul 06 14:24:24.723479 ip-10-0-128-205 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:24:24.723479 ip-10-0-128-205 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 14:24:24.723479 ip-10-0-128-205 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:24:24.723479 ip-10-0-128-205 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 14:24:24.723479 ip-10-0-128-205 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:24:24.724212 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.723559 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 14:24:24.728189 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728163 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:24:24.728189 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728186 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:24:24.728189 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728193 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:24:24.728189 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728197 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728201 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728205 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728210 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728214 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728218 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728221 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728225 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728229 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728233 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728237 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728241 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728245 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728248 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728252 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728257 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728260 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728264 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728268 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:24:24.728370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728272 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728278 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728284 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728288 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728293 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728297 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728301 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728305 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728309 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728313 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728318 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728322 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728326 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728328 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728331 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728335 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728337 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728340 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728343 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728345 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:24:24.728877 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728348 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728350 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728353 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728355 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728357 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728361 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728364 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728367 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728369 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728371 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728374 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728377 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728379 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728382 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728384 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728388 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728392 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728395 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728398 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:24:24.729353 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728400 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728403 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728406 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728408 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728411 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728413 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728416 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728418 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728422 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728425 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728428 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728430 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728433 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728435 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728437 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728440 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728443 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728445 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728448 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:24:24.729824 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728450 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728452 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728455 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728461 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728465 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728468 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728470 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728473 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728476 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728478 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728481 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728483 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728486 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728489 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728492 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728495 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728497 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728499 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.728502 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:24:24.730274 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729227 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729234 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729238 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729241 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729244 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729247 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729250 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729252 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729255 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729258 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729261 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729263 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729266 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729268 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729271 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729273 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729276 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729279 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729282 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:24:24.730725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729286 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729291 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729295 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729298 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729301 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729304 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729307 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729310 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729313 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729315 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729318 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729321 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729323 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729326 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729328 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729331 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729334 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729336 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729339 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729342 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:24:24.731235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729345 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729348 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729350 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729353 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729356 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729359 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729362 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729364 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729367 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729369 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729372 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729375 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729378 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729380 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729383 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729385 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729388 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729390 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729392 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729395 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:24:24.731727 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729398 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729402 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729405 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729409 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729412 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729414 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729417 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729420 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729422 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729425 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729428 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729430 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729433 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729435 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729438 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729440 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729443 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729446 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729448 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:24:24.732231 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729451 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729454 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729456 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729458 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729461 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729464 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729466 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729469 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729471 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729474 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729476 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729478 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729481 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729483 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729486 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729488 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729491 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729493 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729496 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729498 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:24:24.732688 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729503 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729901 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729906 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729909 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729912 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729914 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729917 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729919 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729922 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729925 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729928 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729930 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729933 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729935 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729938 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729940 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729942 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729947 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729951 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:24:24.733184 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729954 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729956 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729959 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729962 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729965 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729967 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729970 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729972 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729975 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729977 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729979 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729982 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729984 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729989 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729991 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729994 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729996 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.729999 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730001 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:24:24.733639 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730004 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730006 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730009 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730012 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730014 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730017 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730021 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730024 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730027 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730030 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730032 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730035 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730038 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730040 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730043 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730046 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730049 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730051 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730054 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730057 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:24:24.734104 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730059 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730062 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730064 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730066 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730069 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730071 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730075 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730077 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730080 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730083 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730085 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730088 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730090 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730093 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730095 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730098 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730101 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730103 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730106 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730108 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:24:24.734575 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730111 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730113 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730116 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730118 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730121 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730124 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730127 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730129 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730131 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730134 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730137 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730142 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730144 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730147 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730149 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730152 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730155 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730157 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730161 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730164 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:24:24.735052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730167 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730169 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730249 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730262 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730272 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730278 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730283 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730286 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730291 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730295 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730298 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730301 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730304 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730308 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730311 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730314 2565 flags.go:64] FLAG: --cgroup-root="" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730317 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730320 2565 flags.go:64] FLAG: --client-ca-file="" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730330 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730333 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730336 2565 flags.go:64] FLAG: --cluster-domain="" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730339 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730342 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730345 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 14:24:24.735538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730348 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730356 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730358 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730361 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730364 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730367 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730371 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730374 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730377 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730380 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730384 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730387 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730390 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730392 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730395 2565 flags.go:64] FLAG: --enable-server="true" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730400 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730405 2565 flags.go:64] FLAG: --event-burst="100" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730408 2565 flags.go:64] FLAG: --event-qps="50" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730411 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730413 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730416 2565 flags.go:64] FLAG: --eviction-hard="" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730420 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730423 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730425 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730428 2565 flags.go:64] FLAG: --eviction-soft="" Jul 06 14:24:24.736124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730431 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730434 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730437 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730440 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730443 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730445 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730448 2565 flags.go:64] FLAG: --feature-gates="" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730452 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730455 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730457 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730460 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730463 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730466 2565 flags.go:64] FLAG: --help="false" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730470 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-128-205.ec2.internal" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730473 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730476 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730478 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730482 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730485 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730487 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730490 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730493 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730496 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 14:24:24.736727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730499 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730502 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730505 2565 flags.go:64] FLAG: --kube-reserved="" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730507 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730510 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730513 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730516 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730518 2565 flags.go:64] FLAG: --lock-file="" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730521 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730524 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730527 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730532 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730535 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730537 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730540 2565 flags.go:64] FLAG: --logging-format="text" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730543 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730546 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730549 2565 flags.go:64] FLAG: --manifest-url="" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730551 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730555 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730558 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730562 2565 flags.go:64] FLAG: --max-pods="110" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730566 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730568 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730571 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 14:24:24.737424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730574 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730577 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730579 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730582 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730589 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730593 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730597 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730600 2565 flags.go:64] FLAG: --pod-cidr="" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730603 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730608 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730611 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730614 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730616 2565 flags.go:64] FLAG: --port="10250" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730619 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730622 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-065df0c33efa42cd6" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730625 2565 flags.go:64] FLAG: --qos-reserved="" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730628 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730631 2565 flags.go:64] FLAG: --register-node="true" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730634 2565 flags.go:64] FLAG: --register-with-taints="" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730637 2565 flags.go:64] FLAG: --registry-burst="10" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730641 2565 flags.go:64] FLAG: --registry-qps="5" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730644 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730647 2565 flags.go:64] FLAG: --reserved-memory="" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730655 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730658 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 14:24:24.738043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730660 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730663 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730666 2565 flags.go:64] FLAG: --runonce="false" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730669 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730672 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730676 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730679 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730682 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730685 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730688 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730691 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730693 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730696 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730700 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730702 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730705 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730708 2565 flags.go:64] FLAG: --system-cgroups="" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730711 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730716 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730719 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730722 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730725 2565 flags.go:64] FLAG: --tls-min-version="" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730729 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730732 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730734 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 14:24:24.738665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730737 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730740 2565 flags.go:64] FLAG: --v="2" Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730745 2565 flags.go:64] FLAG: --version="false" Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730749 2565 flags.go:64] FLAG: --vmodule="" Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730753 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.730756 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730890 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730895 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730898 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730902 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730904 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730907 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730910 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730913 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730916 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730918 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730921 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730924 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730927 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730930 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730934 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:24:24.739282 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730937 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730939 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730942 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730944 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730947 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730949 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730952 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730954 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730957 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730959 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730962 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730964 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730967 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730969 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730971 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730974 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730977 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730979 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730982 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730984 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:24:24.739816 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730986 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730989 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730991 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730993 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730996 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.730999 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731001 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731005 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731008 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731010 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731013 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731016 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731019 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731021 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731024 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731027 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731029 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731033 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731036 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:24:24.740346 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731039 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731042 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731044 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731046 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731049 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731052 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731055 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731058 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731060 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731064 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731066 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731069 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731071 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731074 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731076 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731079 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731082 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731084 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731087 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731089 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:24:24.740830 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731092 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731094 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731096 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731099 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731102 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731105 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731107 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731109 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731112 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731114 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731117 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731119 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731122 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731124 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731127 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731131 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731134 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731137 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731139 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:24:24.741298 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731141 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731144 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731146 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731149 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731152 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.731155 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.731643 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.739176 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.739191 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.739203 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.739210 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739268 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739272 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739276 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739279 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:24:24.741759 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739282 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739285 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739288 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739291 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739294 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739297 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739300 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739303 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739306 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739308 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739311 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739313 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739316 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739318 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739321 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739323 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739326 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739329 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739331 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739334 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:24:24.742134 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739337 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739340 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739343 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739346 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739349 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739351 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739355 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739358 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739360 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739363 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739365 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739368 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739370 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739373 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739376 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739379 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739381 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739384 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739387 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739389 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:24:24.742609 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739392 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739394 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739397 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739399 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739402 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739405 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739407 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739410 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739412 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739416 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739420 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739423 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739425 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739428 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739431 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739434 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739436 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739439 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739442 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:24:24.743098 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739445 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739447 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739450 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739452 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739455 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739458 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739461 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739464 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739466 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739469 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739471 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739473 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739476 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739479 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739481 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739483 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739486 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739488 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739491 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739493 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:24:24.743589 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739495 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739498 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739500 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739502 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739505 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739507 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739510 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739513 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739517 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739521 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739524 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739528 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739530 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739533 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739535 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:24:24.744081 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739538 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.739544 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739678 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739682 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739685 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739688 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739691 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739694 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739697 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739700 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739702 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739705 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739707 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739710 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739714 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:24:24.744449 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739717 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739719 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739722 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739724 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739726 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739729 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739731 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739733 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739736 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739739 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739741 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739745 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739748 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739750 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739753 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739755 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739758 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739760 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739762 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739765 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:24:24.744823 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739767 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739770 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739772 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739774 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739778 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739802 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739806 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739810 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739813 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739816 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739819 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739822 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739825 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739827 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739830 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739833 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739835 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739838 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739840 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:24:24.745307 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739850 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739853 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739856 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739858 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739863 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739865 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739868 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739871 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739874 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739876 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739878 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739881 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739883 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739886 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739888 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739890 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739893 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739895 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739898 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739900 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:24:24.745756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739902 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739905 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739907 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739910 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739912 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739914 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739917 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739919 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739922 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739925 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739927 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739930 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739932 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739934 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739937 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739939 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739942 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739945 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739947 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:24:24.746235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739950 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739952 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739955 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739957 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739959 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739962 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739964 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:24.739966 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.739971 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.740075 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.742759 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.744114 2565 server.go:1019] "Starting client certificate rotation" Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.744218 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:24:24.746680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.744881 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:24:24.749675 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.749660 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:24:24.753256 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.753100 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:24:24.769817 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.769775 2565 log.go:25] "Validated CRI v1 runtime API" Jul 06 14:24:24.771742 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.771724 2565 log.go:25] "Validated CRI v1 image API" Jul 06 14:24:24.772485 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.772458 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 14:24:24.774915 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.774888 2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a3b47252-111d-4302-ab2d-a17c145e972b:/dev/nvme0n1p3 d2c12908-b666-413d-8430-894f66836bcf:/dev/nvme0n1p4] Jul 06 14:24:24.774991 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.774913 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 14:24:24.779509 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.779492 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:24:24.781099 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.780990 2565 manager.go:216] Machine: {Timestamp:2026-07-06 14:24:24.779913426 +0000 UTC m=+0.322600956 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102132 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2923b4f29b37bfe41ba1197a38c5a5 SystemUUID:ec2923b4-f29b-37bf-e41b-a1197a38c5a5 BootID:c0301cc6-74c0-447a-86d9-2140b221345c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cd:f7:32:6b:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cd:f7:32:6b:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:55:96:16:35:71 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 14:24:24.781099 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.781097 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 14:24:24.781198 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.781179 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 14:24:24.783858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.783823 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 14:24:24.784017 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.783862 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-205.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 14:24:24.784487 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.784478 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 14:24:24.784516 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.784490 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 14:24:24.784516 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.784497 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:24:24.785160 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.785149 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:24:24.786285 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.786275 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 14:24:24.789024 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789012 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:24:24.789150 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789141 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 14:24:24.789191 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789171 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 06 14:24:24.789191 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789189 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 14:24:24.789258 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789204 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 14:24:24.789258 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789215 2565 kubelet.go:398] "Adding apiserver pod source" Jul 06 14:24:24.789258 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789225 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 14:24:24.789656 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789641 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:24:24.789699 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.789661 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:24:24.792229 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.792206 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 14:24:24.793825 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.793810 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 14:24:24.793886 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.793842 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 14:24:24.794886 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.794872 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 14:24:24.794966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.794930 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 14:24:24.794966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.794941 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 14:24:24.794966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.794954 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 14:24:24.795911 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.795804 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 14:24:24.796036 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.796017 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 14:24:24.796131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.796042 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 14:24:24.796131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.796055 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 14:24:24.796131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.796068 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 14:24:24.796131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.796079 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 14:24:24.796131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.796109 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 14:24:24.796131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.796123 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 14:24:24.797207 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.797185 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 14:24:24.797207 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.797211 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 14:24:24.799648 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.799626 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-205.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 14:24:24.799752 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.799717 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 14:24:24.799803 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.799736 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-205.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 14:24:24.800820 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.800805 2565 server.go:1268] "Started kubelet" Jul 06 14:24:24.800919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.800881 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 14:24:24.800968 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.800900 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 14:24:24.801019 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.800972 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 14:24:24.801737 ip-10-0-128-205 systemd[1]: Started Kubernetes Kubelet. Jul 06 14:24:24.802365 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.802350 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 14:24:24.803493 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.803479 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 06 14:24:24.806037 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.805107 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-205.ec2.internal.18bfb96f38667c67 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-205.ec2.internal,UID:ip-10-0-128-205.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-205.ec2.internal,},FirstTimestamp:2026-07-06 14:24:24.800754791 +0000 UTC m=+0.343442320,LastTimestamp:2026-07-06 14:24:24.800754791 +0000 UTC m=+0.343442320,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-205.ec2.internal,}" Jul 06 14:24:24.806189 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.806171 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 14:24:24.809326 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.809308 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 14:24:24.809410 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.809311 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 14:24:24.809449 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.809410 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:24.809740 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.809722 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 14:24:24.809837 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.809801 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 14:24:24.809837 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.809823 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 14:24:24.809953 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.809895 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 06 14:24:24.810226 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.810210 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 06 14:24:24.810535 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.810515 2565 factory.go:55] Registering systemd factory Jul 06 14:24:24.810535 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.810536 2565 factory.go:223] Registration of the systemd container factory successfully Jul 06 14:24:24.810772 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.810758 2565 factory.go:153] Registering CRI-O factory Jul 06 14:24:24.810860 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.810775 2565 factory.go:223] Registration of the crio container factory successfully Jul 06 14:24:24.810860 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.810841 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 14:24:24.810969 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.810868 2565 factory.go:103] Registering Raw factory Jul 06 14:24:24.810969 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.810883 2565 manager.go:1196] Started watching for new ooms in manager Jul 06 14:24:24.811804 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.811503 2565 manager.go:349] Starting recovery of all containers Jul 06 14:24:24.812285 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.812254 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vw9t8" Jul 06 14:24:24.821382 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.821352 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vw9t8" Jul 06 14:24:24.821770 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.821723 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-205.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 14:24:24.821956 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.821915 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 14:24:24.823049 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.822875 2565 manager.go:354] Recovery completed Jul 06 14:24:24.828637 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.828621 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:24:24.830392 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.830374 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:24:24.830456 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.830404 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:24:24.830456 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.830414 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:24:24.831042 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831024 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 14:24:24.831042 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831040 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 14:24:24.831166 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831057 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:24:24.831166 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831059 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:24:24.831687 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831673 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:24:24.831750 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831700 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:24:24.831750 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831713 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:24:24.831963 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831953 2565 policy_none.go:49] "None policy: Start" Jul 06 14:24:24.832004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831969 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 14:24:24.832004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.831978 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 14:24:24.832813 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.832803 2565 policy_none.go:47] "Start" Jul 06 14:24:24.875404 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.875383 2565 manager.go:337] "Starting Device Plugin manager" Jul 06 14:24:24.875534 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.875418 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 14:24:24.875534 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.875428 2565 server.go:87] "Starting device plugin registration server" Jul 06 14:24:24.875703 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.875690 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 14:24:24.875741 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.875708 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 14:24:24.875833 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.875819 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 14:24:24.875917 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.875906 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 14:24:24.875990 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.875918 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 14:24:24.876458 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.876438 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 14:24:24.876537 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.876493 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:24.924704 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.924674 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 14:24:24.925959 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.925950 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 14:24:24.926039 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.925983 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 14:24:24.926039 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.926005 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 14:24:24.926133 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.926041 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 14:24:24.928193 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.928171 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:24:24.976315 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.976237 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:24:24.977214 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.977197 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:24:24.977308 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.977231 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:24:24.977308 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.977241 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:24:24.977308 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.977267 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-205.ec2.internal" Jul 06 14:24:24.985550 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:24.985532 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-205.ec2.internal" Jul 06 14:24:24.985635 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:24.985556 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-205.ec2.internal\": node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.002550 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.002528 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.026155 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.026127 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal"] Jul 06 14:24:25.026226 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.026192 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:24:25.027239 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.027223 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:24:25.027315 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.027254 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:24:25.027315 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.027263 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:24:25.028631 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.028618 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:24:25.028764 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.028739 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.028841 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.028769 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:24:25.029334 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.029315 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:24:25.029334 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.029326 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:24:25.029456 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.029347 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:24:25.029456 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.029347 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:24:25.029456 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.029368 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:24:25.029456 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.029357 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:24:25.030826 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.030811 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.030884 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.030842 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:24:25.031558 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.031542 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:24:25.031629 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.031572 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:24:25.031629 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.031584 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:24:25.055553 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.055528 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-205.ec2.internal\" not found" node="ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.059991 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.059972 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-205.ec2.internal\" not found" node="ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.103502 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.103473 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.203774 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.203745 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.211035 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.211004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30b86a1da8fd15e905d0105a81fb274e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal\" (UID: \"30b86a1da8fd15e905d0105a81fb274e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.211035 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.211032 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30b86a1da8fd15e905d0105a81fb274e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal\" (UID: \"30b86a1da8fd15e905d0105a81fb274e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.211160 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.211061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbc6eb3049ab39b232b7d93c6478c5b8-config\") pod \"kube-apiserver-proxy-ip-10-0-128-205.ec2.internal\" (UID: \"fbc6eb3049ab39b232b7d93c6478c5b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.304527 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.304433 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.311744 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.311720 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30b86a1da8fd15e905d0105a81fb274e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal\" (UID: \"30b86a1da8fd15e905d0105a81fb274e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.311858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.311759 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbc6eb3049ab39b232b7d93c6478c5b8-config\") pod \"kube-apiserver-proxy-ip-10-0-128-205.ec2.internal\" (UID: \"fbc6eb3049ab39b232b7d93c6478c5b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.311858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.311775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30b86a1da8fd15e905d0105a81fb274e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal\" (UID: \"30b86a1da8fd15e905d0105a81fb274e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.311858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.311815 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30b86a1da8fd15e905d0105a81fb274e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal\" (UID: \"30b86a1da8fd15e905d0105a81fb274e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.311858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.311847 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30b86a1da8fd15e905d0105a81fb274e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal\" (UID: \"30b86a1da8fd15e905d0105a81fb274e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.312004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.311849 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbc6eb3049ab39b232b7d93c6478c5b8-config\") pod \"kube-apiserver-proxy-ip-10-0-128-205.ec2.internal\" (UID: \"fbc6eb3049ab39b232b7d93c6478c5b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.358854 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.358828 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.363909 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.363656 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" Jul 06 14:24:25.404754 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.404716 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.505202 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.505168 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.605755 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.605660 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.663707 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.663676 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:24:25.706754 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.706726 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.743813 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.743777 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 14:24:25.744313 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.743939 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 14:24:25.744313 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.743980 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 14:24:25.806911 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.806881 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.824129 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.824072 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 14:19:25 +0000 UTC" deadline="2028-03-26 15:23:37.35084465 +0000 UTC" Jul 06 14:24:25.824129 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.824114 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15096h59m11.526735124s" Jul 06 14:24:25.907406 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:25.907327 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:25.924248 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:25.924207 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b86a1da8fd15e905d0105a81fb274e.slice/crio-6f46adc545dd2435544820f5e0855f529b8ce683ce1288a5829f2b98e6a36cac WatchSource:0}: Error finding container 6f46adc545dd2435544820f5e0855f529b8ce683ce1288a5829f2b98e6a36cac: Status 404 returned error can't find the container with id 6f46adc545dd2435544820f5e0855f529b8ce683ce1288a5829f2b98e6a36cac Jul 06 14:24:25.931244 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.931200 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" event={"ID":"30b86a1da8fd15e905d0105a81fb274e","Type":"ContainerStarted","Data":"6f46adc545dd2435544820f5e0855f529b8ce683ce1288a5829f2b98e6a36cac"} Jul 06 14:24:25.931631 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:25.931618 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:24:26.007613 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:26.007560 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:26.074368 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:26.074336 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc6eb3049ab39b232b7d93c6478c5b8.slice/crio-ee0885a1d880b3ba222c4f6cb2e8d876dfa08fb2f7d6f3bfda96dfc7f53a22c2 WatchSource:0}: Error finding container ee0885a1d880b3ba222c4f6cb2e8d876dfa08fb2f7d6f3bfda96dfc7f53a22c2: Status 404 returned error can't find the container with id ee0885a1d880b3ba222c4f6cb2e8d876dfa08fb2f7d6f3bfda96dfc7f53a22c2 Jul 06 14:24:26.108589 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:26.108546 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:26.210246 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:26.210202 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-205.ec2.internal\" not found" Jul 06 14:24:26.232900 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.232870 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:24:26.310503 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.310452 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" Jul 06 14:24:26.323422 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.323392 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:24:26.324380 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.324366 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" Jul 06 14:24:26.334425 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.334402 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:24:26.425800 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.425734 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:24:26.792205 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.792176 2565 apiserver.go:52] "Watching apiserver" Jul 06 14:24:26.805653 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.805623 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 14:24:26.806032 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.806008 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-rw6gh","kube-system/global-pull-secret-syncer-rjl52","kube-system/konnectivity-agent-f9s5h","kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg","openshift-multus/multus-7n788","openshift-multus/multus-additional-cni-plugins-nd4wx","openshift-multus/network-metrics-daemon-wvgxw","openshift-network-diagnostics/network-check-target-9ls2q","openshift-cluster-node-tuning-operator/tuned-c5vxr","openshift-dns/node-resolver-5rtnh","openshift-image-registry/node-ca-kfnjw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal","openshift-network-operator/iptables-alerter-mzvkd"] Jul 06 14:24:26.808668 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.808642 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.810049 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.810025 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 14:24:26.810971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.810950 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.811989 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.811967 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:24:26.812096 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.812084 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 14:24:26.812383 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.812285 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vzzgw\"" Jul 06 14:24:26.813490 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.813178 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:26.814290 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.814273 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-skqzn\"" Jul 06 14:24:26.814834 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.814481 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 14:24:26.814834 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.814726 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 14:24:26.815486 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.815452 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:26.816320 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.816119 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 14:24:26.816320 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.816144 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wh6cl\"" Jul 06 14:24:26.817094 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.816985 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 14:24:26.818263 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.817880 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:26.818457 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818422 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-kubernetes\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818549 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818469 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysctl-conf\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818549 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818498 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-lib-modules\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818549 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818524 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-var-lib-kubelet\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818671 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818573 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-run\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818671 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818619 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-hosts-file\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.818671 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818658 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-tmp-dir\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.818797 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818695 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-modprobe-d\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818797 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-sys\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818797 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-host\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818797 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818766 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-tuned\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.818797 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818784 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrmc8\" (UniqueName: \"kubernetes.io/projected/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-kube-api-access-rrmc8\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.819046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818839 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysconfig\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.819046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysctl-d\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.819046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818888 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-systemd\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.819046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-tmp\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.819046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818947 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:24:26.819046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818968 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6t5n\" (UniqueName: \"kubernetes.io/projected/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-kube-api-access-l6t5n\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.819046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818987 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 14:24:26.819046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.818996 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pqgxw\"" Jul 06 14:24:26.819439 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.819303 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 14:24:26.820300 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.820280 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7n788" Jul 06 14:24:26.821002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.820980 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hdqfq\"" Jul 06 14:24:26.821125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.821108 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 14:24:26.821177 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.821142 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 14:24:26.821227 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.821181 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 14:24:26.822931 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.822762 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:26.822931 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:26.822877 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:26.823044 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.822938 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 14:24:26.823374 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.823355 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 14:24:26.823463 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.823398 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 14:24:26.823519 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.823398 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 14:24:26.823824 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.823805 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f9m7z\"" Jul 06 14:24:26.825200 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.825131 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:26.825279 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:26.825252 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:26.827852 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.827829 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:26.827946 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:26.827902 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:26.829105 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.829086 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:24:26.830250 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.830226 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.833135 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.833113 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 14:24:26.833594 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.833570 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 14:24:26.833687 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.833629 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5thkm\"" Jul 06 14:24:26.833812 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.833797 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.833922 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.833895 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 14:24:26.834276 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.834257 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 14:24:26.834590 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.834572 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 14:24:26.837681 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.837659 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:26.839585 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.839389 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 14:24:26.839585 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.839448 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 14:24:26.839585 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.839460 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vgmrw\"" Jul 06 14:24:26.840660 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.840637 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c9xdf\"" Jul 06 14:24:26.840762 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.840706 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 14:24:26.840939 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.840874 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 14:24:26.840939 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.840937 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 14:24:26.841652 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.841635 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 14:24:26.846271 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.846246 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-srtgs" Jul 06 14:24:26.855056 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.855025 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-srtgs" Jul 06 14:24:26.911610 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.911579 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 14:24:26.919399 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919364 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-tmp-dir\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.919521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919406 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:26.919521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919431 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-ovnkube-config\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.919521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919455 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-cni-bin\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.919521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919471 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsblb\" (UniqueName: \"kubernetes.io/projected/d7d6be71-c596-4932-a6a3-ffdacda9a326-kube-api-access-nsblb\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.919521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919489 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p67t7\" (UniqueName: \"kubernetes.io/projected/8ca6503a-77c2-47ef-8587-531b6ee00e34-kube-api-access-p67t7\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:26.919521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919511 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-sys\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919532 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-node-log\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919548 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-hostroot\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919562 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-conf-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-system-cni-dir\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919593 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-os-release\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919621 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysctl-d\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919644 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-systemd\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919694 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6t5n\" (UniqueName: \"kubernetes.io/projected/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-kube-api-access-l6t5n\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-slash\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.919743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919721 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-tmp-dir\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919726 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzk94\" (UniqueName: \"kubernetes.io/projected/0c54d035-2286-4800-81cd-c71c8ff0e373-kube-api-access-jzk94\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919792 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-device-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919826 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-sys\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919875 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919878 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-systemd\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919931 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-run-netns\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.919955 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-etc-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920000 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-log-socket\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920011 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysctl-d\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-ovnkube-script-lib\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l7fl\" (UniqueName: \"kubernetes.io/projected/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-kube-api-access-9l7fl\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:26.920079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjkfc\" (UniqueName: \"kubernetes.io/projected/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-kube-api-access-fjkfc\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-run\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920127 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fcf896e5-cbff-43fa-8d73-b05d9be1a499-dbus\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920152 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-systemd-units\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920192 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-cni-bin\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920215 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65638096-89bf-4730-b736-4412ccfd6c62-agent-certs\") pod \"konnectivity-agent-f9s5h\" (UID: \"65638096-89bf-4730-b736-4412ccfd6c62\") " pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920261 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65638096-89bf-4730-b736-4412ccfd6c62-konnectivity-ca\") pod \"konnectivity-agent-f9s5h\" (UID: \"65638096-89bf-4730-b736-4412ccfd6c62\") " pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920279 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-iptables-alerter-script\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920300 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-cni-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920219 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-run\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920317 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-modprobe-d\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920342 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-env-overrides\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920383 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-cni-multus\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920459 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-kubelet\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920491 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-modprobe-d\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920493 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfjql\" (UniqueName: \"kubernetes.io/projected/958ab8c5-dc0d-4a1e-a602-947c2a199009-kube-api-access-xfjql\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920548 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-os-release\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.920617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920583 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ca6503a-77c2-47ef-8587-531b6ee00e34-host\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ca6503a-77c2-47ef-8587-531b6ee00e34-serviceca\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysctl-conf\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920711 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-lib-modules\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920754 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-kubelet\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920776 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysctl-conf\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920809 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-systemd\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920837 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-sys-fs\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920896 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-lib-modules\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920916 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-k8s-cni-cncf-io\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-hosts-file\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.920991 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-socket-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921015 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-hosts-file\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921018 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rdtp\" (UniqueName: \"kubernetes.io/projected/03b14c6a-25bf-470e-8495-4848f20e3728-kube-api-access-4rdtp\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921046 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-multus-certs\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.921543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921090 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-cni-binary-copy\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrmc8\" (UniqueName: \"kubernetes.io/projected/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-kube-api-access-rrmc8\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921227 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c54d035-2286-4800-81cd-c71c8ff0e373-ovn-node-metrics-cert\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921283 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fcf896e5-cbff-43fa-8d73-b05d9be1a499-kubelet-config\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921305 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-ovn\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921348 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-host-slash\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921371 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921395 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-kubernetes\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-var-lib-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921464 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-kubernetes\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921454 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-run-ovn-kubernetes\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921511 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-system-cni-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921528 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-cnibin\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921544 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-etc-selinux\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:26.922268 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921622 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-etc-kubernetes\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921652 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-host\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-tuned\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921704 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921730 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-socket-dir-parent\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921756 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-netns\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921759 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-host\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921782 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysconfig\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921862 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-sysconfig\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921837 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-tmp\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-cni-netd\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921929 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-registration-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.921949 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7d6be71-c596-4932-a6a3-ffdacda9a326-cni-binary-copy\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.922004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-daemon-config\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.922039 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.922041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-var-lib-kubelet\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.922089 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-var-lib-kubelet\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.923081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.922109 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-cnibin\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:26.923834 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.922130 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:26.925543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.925515 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-etc-tuned\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.925659 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.925558 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-tmp\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.930555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.930252 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6t5n\" (UniqueName: \"kubernetes.io/projected/3e3a655e-a83a-43a2-b610-6b66bf7c6fc0-kube-api-access-l6t5n\") pod \"node-resolver-5rtnh\" (UID: \"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0\") " pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:26.930555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.930500 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrmc8\" (UniqueName: \"kubernetes.io/projected/c4b84429-56d3-4e13-ba69-a8fa3474b2b0-kube-api-access-rrmc8\") pod \"tuned-c5vxr\" (UID: \"c4b84429-56d3-4e13-ba69-a8fa3474b2b0\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:26.937373 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:26.937343 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" event={"ID":"fbc6eb3049ab39b232b7d93c6478c5b8","Type":"ContainerStarted","Data":"ee0885a1d880b3ba222c4f6cb2e8d876dfa08fb2f7d6f3bfda96dfc7f53a22c2"} Jul 06 14:24:27.022805 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.022766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-node-log\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.022994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.022826 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-hostroot\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.022994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.022852 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-conf-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.022994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.022879 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-system-cni-dir\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.022994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.022904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-os-release\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.022994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.022941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-slash\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.022994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.022968 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzk94\" (UniqueName: \"kubernetes.io/projected/0c54d035-2286-4800-81cd-c71c8ff0e373-kube-api-access-jzk94\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023002 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-device-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023076 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-run-netns\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023103 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-etc-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023129 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-log-socket\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023156 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-ovnkube-script-lib\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023178 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7fl\" (UniqueName: \"kubernetes.io/projected/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-kube-api-access-9l7fl\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023180 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-slash\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjkfc\" (UniqueName: \"kubernetes.io/projected/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-kube-api-access-fjkfc\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:27.023279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fcf896e5-cbff-43fa-8d73-b05d9be1a499-dbus\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:27.023891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023640 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-etc-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023687 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-node-log\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023726 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-hostroot\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.023891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023754 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-conf-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.023891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023761 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-ovnkube-script-lib\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.023891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023778 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-system-cni-dir\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.023891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023844 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-os-release\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.024183 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023899 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-systemd-units\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.024183 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-cni-bin\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.024183 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65638096-89bf-4730-b736-4412ccfd6c62-agent-certs\") pod \"konnectivity-agent-f9s5h\" (UID: \"65638096-89bf-4730-b736-4412ccfd6c62\") " pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:27.024183 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.023967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65638096-89bf-4730-b736-4412ccfd6c62-konnectivity-ca\") pod \"konnectivity-agent-f9s5h\" (UID: \"65638096-89bf-4730-b736-4412ccfd6c62\") " pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:27.024183 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.024012 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-log-socket\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.024428 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.024052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-iptables-alerter-script\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:27.024428 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.024336 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-cni-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.024428 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.024386 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-env-overrides\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.024428 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.024422 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-cni-multus\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.024632 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.024464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-kubelet\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.026020 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.025978 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fcf896e5-cbff-43fa-8d73-b05d9be1a499-dbus\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:27.026369 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.026172 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:27.026369 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026279 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfjql\" (UniqueName: \"kubernetes.io/projected/958ab8c5-dc0d-4a1e-a602-947c2a199009-kube-api-access-xfjql\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.026369 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026315 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-os-release\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.026369 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026345 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ca6503a-77c2-47ef-8587-531b6ee00e34-host\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:27.026369 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026367 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ca6503a-77c2-47ef-8587-531b6ee00e34-serviceca\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:27.026587 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026392 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-kubelet\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.026587 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026415 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-systemd\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.026587 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026434 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.026587 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-sys-fs\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.026587 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026513 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-k8s-cni-cncf-io\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.026738 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026628 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-socket-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.026738 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rdtp\" (UniqueName: \"kubernetes.io/projected/03b14c6a-25bf-470e-8495-4848f20e3728-kube-api-access-4rdtp\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.026738 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-multus-certs\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.026979 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026954 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-cni-binary-copy\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.027060 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.026998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.027060 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027037 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c54d035-2286-4800-81cd-c71c8ff0e373-ovn-node-metrics-cert\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.027162 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fcf896e5-cbff-43fa-8d73-b05d9be1a499-kubelet-config\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:27.027162 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-ovn\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.027162 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-host-slash\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:27.027292 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027180 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.027292 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027218 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-var-lib-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.027292 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-run-ovn-kubernetes\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.027385 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-system-cni-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.027385 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027330 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-cnibin\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.027385 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027367 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.027482 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027405 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-etc-selinux\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.027653 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-etc-kubernetes\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.027712 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027680 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.027770 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-socket-dir-parent\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.027770 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-netns\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.027903 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-cni-netd\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.027903 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027820 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-registration-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.027903 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7d6be71-c596-4932-a6a3-ffdacda9a326-cni-binary-copy\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.028067 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-daemon-config\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.028067 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027958 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-cnibin\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.028067 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.027991 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.028067 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.028025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:27.028067 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.028060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-ovnkube-config\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.028317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.028097 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-cni-bin\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.028317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.028135 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsblb\" (UniqueName: \"kubernetes.io/projected/d7d6be71-c596-4932-a6a3-ffdacda9a326-kube-api-access-nsblb\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.028317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.028174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p67t7\" (UniqueName: \"kubernetes.io/projected/8ca6503a-77c2-47ef-8587-531b6ee00e34-kube-api-access-p67t7\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:27.028614 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.028585 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-etc-selinux\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.028678 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.028646 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-k8s-cni-cncf-io\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.029221 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029193 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-iptables-alerter-script\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:27.029322 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029243 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65638096-89bf-4730-b736-4412ccfd6c62-agent-certs\") pod \"konnectivity-agent-f9s5h\" (UID: \"65638096-89bf-4730-b736-4412ccfd6c62\") " pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:27.029661 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029639 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65638096-89bf-4730-b736-4412ccfd6c62-konnectivity-ca\") pod \"konnectivity-agent-f9s5h\" (UID: \"65638096-89bf-4730-b736-4412ccfd6c62\") " pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:27.029661 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029648 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fcf896e5-cbff-43fa-8d73-b05d9be1a499-kubelet-config\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:27.029807 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029708 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-ovn\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.029807 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029712 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-var-lib-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.029807 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029737 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-run-ovn-kubernetes\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.029807 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029785 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-system-cni-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.030060 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029816 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-cnibin\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.030060 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.029960 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/958ab8c5-dc0d-4a1e-a602-947c2a199009-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.030060 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030002 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-etc-kubernetes\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.030060 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030034 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-openvswitch\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.030278 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030080 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-socket-dir-parent\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.030278 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030111 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-netns\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.030278 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030142 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-cni-netd\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.030278 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030178 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-registration-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.030468 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030310 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-cni-dir\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.030468 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.030398 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs podName:9b04e6a4-fe9b-4b78-954a-aa1927eb521a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:27.530375765 +0000 UTC m=+3.073063299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs") pod "network-metrics-daemon-wvgxw" (UID: "9b04e6a4-fe9b-4b78-954a-aa1927eb521a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:27.030578 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030561 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7d6be71-c596-4932-a6a3-ffdacda9a326-cni-binary-copy\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.030677 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030650 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-cni-multus\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.030928 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030910 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d7d6be71-c596-4932-a6a3-ffdacda9a326-multus-daemon-config\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.031015 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.030985 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-cnibin\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.031076 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031051 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-env-overrides\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.031334 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031313 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-os-release\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.031457 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031432 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ca6503a-77c2-47ef-8587-531b6ee00e34-host\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:27.031457 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.031450 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:27.031559 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.031542 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret podName:fcf896e5-cbff-43fa-8d73-b05d9be1a499 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:27.531515872 +0000 UTC m=+3.074203407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret") pod "global-pull-secret-syncer-rjl52" (UID: "fcf896e5-cbff-43fa-8d73-b05d9be1a499") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:27.031559 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031544 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-kubelet\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.031676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031575 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.031676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031584 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-run-systemd\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.031676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031604 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-cni-bin\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.031676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031644 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.031676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031660 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-cni-bin\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.031983 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031733 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-run-multus-certs\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.031983 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031803 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-sys-fs\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.031983 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031884 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-host-run-netns\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.031983 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.031938 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.032196 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032165 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ca6503a-77c2-47ef-8587-531b6ee00e34-serviceca\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:27.032256 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032223 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.032256 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032242 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c54d035-2286-4800-81cd-c71c8ff0e373-ovnkube-config\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.032256 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032227 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-host-slash\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:27.032256 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032250 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c54d035-2286-4800-81cd-c71c8ff0e373-systemd-units\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.032465 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032298 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-device-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.032465 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032384 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d6be71-c596-4932-a6a3-ffdacda9a326-host-var-lib-kubelet\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.032465 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032394 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/958ab8c5-dc0d-4a1e-a602-947c2a199009-cni-binary-copy\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.032465 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.032384 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03b14c6a-25bf-470e-8495-4848f20e3728-socket-dir\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.033554 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.032845 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:24:27.033554 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.032869 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:24:27.033554 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.032892 2565 projected.go:196] Error preparing data for projected volume kube-api-access-s644p for pod openshift-network-diagnostics/network-check-target-9ls2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:27.033554 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.032973 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p podName:626f1048-dffc-4449-b748-ef625d7de486 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:27.532948412 +0000 UTC m=+3.075635949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s644p" (UniqueName: "kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p") pod "network-check-target-9ls2q" (UID: "626f1048-dffc-4449-b748-ef625d7de486") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:27.035538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.035512 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c54d035-2286-4800-81cd-c71c8ff0e373-ovn-node-metrics-cert\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.036004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.035944 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzk94\" (UniqueName: \"kubernetes.io/projected/0c54d035-2286-4800-81cd-c71c8ff0e373-kube-api-access-jzk94\") pod \"ovnkube-node-rw6gh\" (UID: \"0c54d035-2286-4800-81cd-c71c8ff0e373\") " pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.037404 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.037379 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l7fl\" (UniqueName: \"kubernetes.io/projected/3cb15cbe-aa6e-4e81-b5cd-ba0eade18021-kube-api-access-9l7fl\") pod \"iptables-alerter-mzvkd\" (UID: \"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021\") " pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:27.038093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.038044 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjkfc\" (UniqueName: \"kubernetes.io/projected/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-kube-api-access-fjkfc\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:27.048753 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.048657 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfjql\" (UniqueName: \"kubernetes.io/projected/958ab8c5-dc0d-4a1e-a602-947c2a199009-kube-api-access-xfjql\") pod \"multus-additional-cni-plugins-nd4wx\" (UID: \"958ab8c5-dc0d-4a1e-a602-947c2a199009\") " pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.048753 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.048665 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p67t7\" (UniqueName: \"kubernetes.io/projected/8ca6503a-77c2-47ef-8587-531b6ee00e34-kube-api-access-p67t7\") pod \"node-ca-kfnjw\" (UID: \"8ca6503a-77c2-47ef-8587-531b6ee00e34\") " pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:27.050919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.050873 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rdtp\" (UniqueName: \"kubernetes.io/projected/03b14c6a-25bf-470e-8495-4848f20e3728-kube-api-access-4rdtp\") pod \"aws-ebs-csi-driver-node-225hg\" (UID: \"03b14c6a-25bf-470e-8495-4848f20e3728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.050919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.050897 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsblb\" (UniqueName: \"kubernetes.io/projected/d7d6be71-c596-4932-a6a3-ffdacda9a326-kube-api-access-nsblb\") pod \"multus-7n788\" (UID: \"d7d6be71-c596-4932-a6a3-ffdacda9a326\") " pod="openshift-multus/multus-7n788" Jul 06 14:24:27.122148 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.122100 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" Jul 06 14:24:27.129886 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.129860 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5rtnh" Jul 06 14:24:27.134931 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.134847 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b84429_56d3_4e13_ba69_a8fa3474b2b0.slice/crio-4e3f7acf7f2d64b54f2d4136a31d93febbf545f7d2ec2e8baccd12cabe8cd423 WatchSource:0}: Error finding container 4e3f7acf7f2d64b54f2d4136a31d93febbf545f7d2ec2e8baccd12cabe8cd423: Status 404 returned error can't find the container with id 4e3f7acf7f2d64b54f2d4136a31d93febbf545f7d2ec2e8baccd12cabe8cd423 Jul 06 14:24:27.140165 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.139984 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:27.143204 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.143180 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3a655e_a83a_43a2_b610_6b66bf7c6fc0.slice/crio-8d08aa1d87a14362fe0b2fb19d0b9235bebbfd5f66cb6e59089371288d44dac0 WatchSource:0}: Error finding container 8d08aa1d87a14362fe0b2fb19d0b9235bebbfd5f66cb6e59089371288d44dac0: Status 404 returned error can't find the container with id 8d08aa1d87a14362fe0b2fb19d0b9235bebbfd5f66cb6e59089371288d44dac0 Jul 06 14:24:27.149716 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.149067 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:27.153052 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.152999 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65638096_89bf_4730_b736_4412ccfd6c62.slice/crio-a65557381c0219ff78be5c4f2df2c17398088d74e01230e3190b9416def98fe4 WatchSource:0}: Error finding container a65557381c0219ff78be5c4f2df2c17398088d74e01230e3190b9416def98fe4: Status 404 returned error can't find the container with id a65557381c0219ff78be5c4f2df2c17398088d74e01230e3190b9416def98fe4 Jul 06 14:24:27.156515 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.155786 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzvkd" Jul 06 14:24:27.163770 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.163399 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" Jul 06 14:24:27.164162 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.164134 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c54d035_2286_4800_81cd_c71c8ff0e373.slice/crio-0946b41e2a763c2c88b6dd6efce1fc15fd0c3744652d5455972073d3d87de926 WatchSource:0}: Error finding container 0946b41e2a763c2c88b6dd6efce1fc15fd0c3744652d5455972073d3d87de926: Status 404 returned error can't find the container with id 0946b41e2a763c2c88b6dd6efce1fc15fd0c3744652d5455972073d3d87de926 Jul 06 14:24:27.167392 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.167364 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb15cbe_aa6e_4e81_b5cd_ba0eade18021.slice/crio-34cffba96f4002ddba167965fd6786f6b56f13b46733d43083eb48dc49e3d72b WatchSource:0}: Error finding container 34cffba96f4002ddba167965fd6786f6b56f13b46733d43083eb48dc49e3d72b: Status 404 returned error can't find the container with id 34cffba96f4002ddba167965fd6786f6b56f13b46733d43083eb48dc49e3d72b Jul 06 14:24:27.168919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.168831 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7n788" Jul 06 14:24:27.177370 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.177198 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" Jul 06 14:24:27.180739 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.180707 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b14c6a_25bf_470e_8495_4848f20e3728.slice/crio-f72714dfe288b6798333d3c2260623e6f9cdd4c5ae5a86c3d8fbde1c69bf7823 WatchSource:0}: Error finding container f72714dfe288b6798333d3c2260623e6f9cdd4c5ae5a86c3d8fbde1c69bf7823: Status 404 returned error can't find the container with id f72714dfe288b6798333d3c2260623e6f9cdd4c5ae5a86c3d8fbde1c69bf7823 Jul 06 14:24:27.182536 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.182248 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kfnjw" Jul 06 14:24:27.185452 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.185407 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d6be71_c596_4932_a6a3_ffdacda9a326.slice/crio-0dc8aba0ae0218899257be8e540ddcbd1c5f14c3189f848d2e073691831466d7 WatchSource:0}: Error finding container 0dc8aba0ae0218899257be8e540ddcbd1c5f14c3189f848d2e073691831466d7: Status 404 returned error can't find the container with id 0dc8aba0ae0218899257be8e540ddcbd1c5f14c3189f848d2e073691831466d7 Jul 06 14:24:27.200323 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.200295 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958ab8c5_dc0d_4a1e_a602_947c2a199009.slice/crio-98d6cfb3d5800f45ca00d0b073acb2e0305dc8729ed457798f1ffc3025cdb5b5 WatchSource:0}: Error finding container 98d6cfb3d5800f45ca00d0b073acb2e0305dc8729ed457798f1ffc3025cdb5b5: Status 404 returned error can't find the container with id 98d6cfb3d5800f45ca00d0b073acb2e0305dc8729ed457798f1ffc3025cdb5b5 Jul 06 14:24:27.203480 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:24:27.203453 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca6503a_77c2_47ef_8587_531b6ee00e34.slice/crio-29ab71e332903ca9c87f1d3f6293200991ae30d27bf95790eae3a3b680cf48cb WatchSource:0}: Error finding container 29ab71e332903ca9c87f1d3f6293200991ae30d27bf95790eae3a3b680cf48cb: Status 404 returned error can't find the container with id 29ab71e332903ca9c87f1d3f6293200991ae30d27bf95790eae3a3b680cf48cb Jul 06 14:24:27.238208 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.238176 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:24:27.531521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.531478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:27.531715 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.531585 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:27.531715 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.531676 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:27.531715 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.531704 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:27.531887 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.531757 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs podName:9b04e6a4-fe9b-4b78-954a-aa1927eb521a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:28.531733558 +0000 UTC m=+4.074421090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs") pod "network-metrics-daemon-wvgxw" (UID: "9b04e6a4-fe9b-4b78-954a-aa1927eb521a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:27.531887 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.531782 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret podName:fcf896e5-cbff-43fa-8d73-b05d9be1a499 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:28.531768999 +0000 UTC m=+4.074456531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret") pod "global-pull-secret-syncer-rjl52" (UID: "fcf896e5-cbff-43fa-8d73-b05d9be1a499") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:27.633101 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.632999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:27.633274 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.633165 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:24:27.633274 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.633185 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:24:27.633274 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.633196 2565 projected.go:196] Error preparing data for projected volume kube-api-access-s644p for pod openshift-network-diagnostics/network-check-target-9ls2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:27.633274 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.633256 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p podName:626f1048-dffc-4449-b748-ef625d7de486 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:28.633237213 +0000 UTC m=+4.175924739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s644p" (UniqueName: "kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p") pod "network-check-target-9ls2q" (UID: "626f1048-dffc-4449-b748-ef625d7de486") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:27.858503 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.858461 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:19:27 +0000 UTC" deadline="2028-03-06 09:00:07.095582067 +0000 UTC" Jul 06 14:24:27.858503 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.858495 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14610h35m39.237090667s" Jul 06 14:24:27.927367 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.927288 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:27.927528 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:27.927452 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:27.940398 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.940363 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" event={"ID":"958ab8c5-dc0d-4a1e-a602-947c2a199009","Type":"ContainerStarted","Data":"98d6cfb3d5800f45ca00d0b073acb2e0305dc8729ed457798f1ffc3025cdb5b5"} Jul 06 14:24:27.941688 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.941657 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" event={"ID":"03b14c6a-25bf-470e-8495-4848f20e3728","Type":"ContainerStarted","Data":"f72714dfe288b6798333d3c2260623e6f9cdd4c5ae5a86c3d8fbde1c69bf7823"} Jul 06 14:24:27.942909 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.942879 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzvkd" event={"ID":"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021","Type":"ContainerStarted","Data":"34cffba96f4002ddba167965fd6786f6b56f13b46733d43083eb48dc49e3d72b"} Jul 06 14:24:27.944242 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.944213 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"0946b41e2a763c2c88b6dd6efce1fc15fd0c3744652d5455972073d3d87de926"} Jul 06 14:24:27.945414 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.945392 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" event={"ID":"c4b84429-56d3-4e13-ba69-a8fa3474b2b0","Type":"ContainerStarted","Data":"4e3f7acf7f2d64b54f2d4136a31d93febbf545f7d2ec2e8baccd12cabe8cd423"} Jul 06 14:24:27.946564 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.946528 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kfnjw" event={"ID":"8ca6503a-77c2-47ef-8587-531b6ee00e34","Type":"ContainerStarted","Data":"29ab71e332903ca9c87f1d3f6293200991ae30d27bf95790eae3a3b680cf48cb"} Jul 06 14:24:27.947835 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.947783 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7n788" event={"ID":"d7d6be71-c596-4932-a6a3-ffdacda9a326","Type":"ContainerStarted","Data":"0dc8aba0ae0218899257be8e540ddcbd1c5f14c3189f848d2e073691831466d7"} Jul 06 14:24:27.948953 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.948931 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f9s5h" event={"ID":"65638096-89bf-4730-b736-4412ccfd6c62","Type":"ContainerStarted","Data":"a65557381c0219ff78be5c4f2df2c17398088d74e01230e3190b9416def98fe4"} Jul 06 14:24:27.950062 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:27.950042 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5rtnh" event={"ID":"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0","Type":"ContainerStarted","Data":"8d08aa1d87a14362fe0b2fb19d0b9235bebbfd5f66cb6e59089371288d44dac0"} Jul 06 14:24:28.095193 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:28.095155 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:24:28.541890 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:28.541828 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:28.542073 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:28.541916 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:28.542152 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.542077 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:28.542152 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.542144 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs podName:9b04e6a4-fe9b-4b78-954a-aa1927eb521a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:30.54212384 +0000 UTC m=+6.084811376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs") pod "network-metrics-daemon-wvgxw" (UID: "9b04e6a4-fe9b-4b78-954a-aa1927eb521a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:28.542277 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.542212 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:28.542277 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.542252 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret podName:fcf896e5-cbff-43fa-8d73-b05d9be1a499 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:30.542238061 +0000 UTC m=+6.084925597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret") pod "global-pull-secret-syncer-rjl52" (UID: "fcf896e5-cbff-43fa-8d73-b05d9be1a499") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:28.643120 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:28.643078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:28.643318 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.643261 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:24:28.643318 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.643282 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:24:28.643318 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.643297 2565 projected.go:196] Error preparing data for projected volume kube-api-access-s644p for pod openshift-network-diagnostics/network-check-target-9ls2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:28.643505 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.643358 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p podName:626f1048-dffc-4449-b748-ef625d7de486 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:30.643338876 +0000 UTC m=+6.186026394 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s644p" (UniqueName: "kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p") pod "network-check-target-9ls2q" (UID: "626f1048-dffc-4449-b748-ef625d7de486") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:28.859177 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:28.859074 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:19:27 +0000 UTC" deadline="2027-12-12 23:37:15.017896728 +0000 UTC" Jul 06 14:24:28.859177 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:28.859122 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12585h12m46.158780739s" Jul 06 14:24:28.928040 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:28.928010 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:28.928239 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.928140 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:28.928513 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:28.928491 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:28.928675 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:28.928602 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:29.927706 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:29.927125 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:29.927706 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:29.927358 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:29.986182 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:29.986140 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" event={"ID":"fbc6eb3049ab39b232b7d93c6478c5b8","Type":"ContainerStarted","Data":"4082440f380bf829ac5710c3478e37ea5246839d94b3395c37680e6e2168156a"} Jul 06 14:24:30.559821 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:30.559770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:30.560036 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:30.559859 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:30.560107 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.560035 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:30.560107 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.560097 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs podName:9b04e6a4-fe9b-4b78-954a-aa1927eb521a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:34.560078784 +0000 UTC m=+10.102766325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs") pod "network-metrics-daemon-wvgxw" (UID: "9b04e6a4-fe9b-4b78-954a-aa1927eb521a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:30.560513 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.560491 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:30.560602 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.560554 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret podName:fcf896e5-cbff-43fa-8d73-b05d9be1a499 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:34.560537305 +0000 UTC m=+10.103224824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret") pod "global-pull-secret-syncer-rjl52" (UID: "fcf896e5-cbff-43fa-8d73-b05d9be1a499") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:30.661104 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:30.661012 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:30.661264 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.661212 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:24:30.661264 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.661237 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:24:30.661264 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.661251 2565 projected.go:196] Error preparing data for projected volume kube-api-access-s644p for pod openshift-network-diagnostics/network-check-target-9ls2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:30.661449 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.661327 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p podName:626f1048-dffc-4449-b748-ef625d7de486 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:34.661306298 +0000 UTC m=+10.203993839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s644p" (UniqueName: "kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p") pod "network-check-target-9ls2q" (UID: "626f1048-dffc-4449-b748-ef625d7de486") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:30.927435 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:30.926775 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:30.927435 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.926955 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:30.927435 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:30.926973 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:30.927435 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:30.927081 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:30.993141 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:30.993071 2565 generic.go:356] "Generic (PLEG): container finished" podID="30b86a1da8fd15e905d0105a81fb274e" containerID="2063041884ff5ba22712cb1cdb31b08ef6625cfde6143b5671e2a740c79f79f3" exitCode=0 Jul 06 14:24:30.994091 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:30.994060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" event={"ID":"30b86a1da8fd15e905d0105a81fb274e","Type":"ContainerDied","Data":"2063041884ff5ba22712cb1cdb31b08ef6625cfde6143b5671e2a740c79f79f3"} Jul 06 14:24:31.012838 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:31.012480 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-205.ec2.internal" podStartSLOduration=5.012459349 podStartE2EDuration="5.012459349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:30.004846844 +0000 UTC m=+5.547534387" watchObservedRunningTime="2026-07-06 14:24:31.012459349 +0000 UTC m=+6.555146894" Jul 06 14:24:31.926971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:31.926919 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:31.927131 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:31.927052 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:32.927766 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:32.927730 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:32.928185 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:32.927743 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:32.928185 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:32.927888 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:32.928185 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:32.927997 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:33.927532 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:33.927102 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:33.927532 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:33.927223 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:34.597996 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:34.597964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:34.598416 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:34.598014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:34.598416 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.598146 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:34.598416 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.598210 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs podName:9b04e6a4-fe9b-4b78-954a-aa1927eb521a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:42.598190052 +0000 UTC m=+18.140877575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs") pod "network-metrics-daemon-wvgxw" (UID: "9b04e6a4-fe9b-4b78-954a-aa1927eb521a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:34.598608 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.598591 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:34.598656 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.598646 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret podName:fcf896e5-cbff-43fa-8d73-b05d9be1a499 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:42.598631303 +0000 UTC m=+18.141318834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret") pod "global-pull-secret-syncer-rjl52" (UID: "fcf896e5-cbff-43fa-8d73-b05d9be1a499") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:34.698828 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:34.698732 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:34.698988 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.698935 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:24:34.698988 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.698960 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:24:34.698988 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.698973 2565 projected.go:196] Error preparing data for projected volume kube-api-access-s644p for pod openshift-network-diagnostics/network-check-target-9ls2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:34.699158 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.699032 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p podName:626f1048-dffc-4449-b748-ef625d7de486 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:42.699018662 +0000 UTC m=+18.241706178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s644p" (UniqueName: "kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p") pod "network-check-target-9ls2q" (UID: "626f1048-dffc-4449-b748-ef625d7de486") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:34.929515 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:34.929436 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:34.929663 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.929538 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:34.929936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:34.929892 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:34.930031 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:34.930004 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:35.926887 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:35.926853 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:35.927252 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:35.926968 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:36.927966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:36.927931 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:36.928467 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:36.927984 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:36.928467 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:36.928102 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:36.928467 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:36.928245 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:37.926403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:37.926368 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:37.926582 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:37.926506 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:38.929063 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:38.929023 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:38.929586 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:38.929148 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:38.929586 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:38.929340 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:38.929586 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:38.929144 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:39.017127 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:39.017096 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5rtnh" event={"ID":"3e3a655e-a83a-43a2-b610-6b66bf7c6fc0","Type":"ContainerStarted","Data":"70895ec5e97edfd740f9d66241abdc8d5599b5bb9857fdcba13c2f8b668b1794"} Jul 06 14:24:39.025650 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:39.025611 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" event={"ID":"30b86a1da8fd15e905d0105a81fb274e","Type":"ContainerStarted","Data":"209d278ecdcc98425bc913e200a7e19070c8765b2cadaf8ce3aa744fd392464e"} Jul 06 14:24:39.038487 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:39.038436 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5rtnh" podStartSLOduration=2.62017763 podStartE2EDuration="14.038415939s" totalImagesPullingTime="11.418238309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:39.036431345 +0000 UTC m=+14.579118879" watchObservedRunningTime="2026-07-06 14:24:39.038415939 +0000 UTC m=+14.581103478" Jul 06 14:24:39.060855 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:39.059313 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-205.ec2.internal" podStartSLOduration=13.059290065 podStartE2EDuration="13.059290065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:39.059038132 +0000 UTC m=+14.601725671" watchObservedRunningTime="2026-07-06 14:24:39.059290065 +0000 UTC m=+14.601977604" Jul 06 14:24:39.927096 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:39.926804 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:39.927269 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:39.927166 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:40.029684 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.029650 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" event={"ID":"c4b84429-56d3-4e13-ba69-a8fa3474b2b0","Type":"ContainerStarted","Data":"831b30368fe99ca517238dafe1a1a1b58a9dac029df0ed2ad789f86cccc910b7"} Jul 06 14:24:40.031304 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.031272 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kfnjw" event={"ID":"8ca6503a-77c2-47ef-8587-531b6ee00e34","Type":"ContainerStarted","Data":"6b2ea5f7dd659df99c4408022a79b91ed3be2a3f9a361f51643c0ecdb6022245"} Jul 06 14:24:40.032754 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.032727 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f9s5h" event={"ID":"65638096-89bf-4730-b736-4412ccfd6c62","Type":"ContainerStarted","Data":"3b2f79086e3ba1c7ee0c0ae8866ef19a48a36d8960a1692e4932ef5f55a3888b"} Jul 06 14:24:40.034403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.034378 2565 generic.go:356] "Generic (PLEG): container finished" podID="958ab8c5-dc0d-4a1e-a602-947c2a199009" containerID="3d6bb60e89225e371ce7a24b1e566f7525f2d418f1494f5ff69a895b4b6db99d" exitCode=0 Jul 06 14:24:40.034505 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.034467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" event={"ID":"958ab8c5-dc0d-4a1e-a602-947c2a199009","Type":"ContainerDied","Data":"3d6bb60e89225e371ce7a24b1e566f7525f2d418f1494f5ff69a895b4b6db99d"} Jul 06 14:24:40.036493 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.036466 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" event={"ID":"03b14c6a-25bf-470e-8495-4848f20e3728","Type":"ContainerStarted","Data":"21cee3783e3d2b099ff16fe511bf9dbfb9a133a3908d60a20e7d978f107cbd57"} Jul 06 14:24:40.051626 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.051585 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c5vxr" podStartSLOduration=3.216870181 podStartE2EDuration="15.051572055s" totalImagesPullingTime="11.834701874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:40.051065948 +0000 UTC m=+15.593753487" watchObservedRunningTime="2026-07-06 14:24:40.051572055 +0000 UTC m=+15.594259592" Jul 06 14:24:40.069043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.068998 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-f9s5h" podStartSLOduration=3.57706944 podStartE2EDuration="15.068980903s" totalImagesPullingTime="11.491911463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:40.068298604 +0000 UTC m=+15.610986146" watchObservedRunningTime="2026-07-06 14:24:40.068980903 +0000 UTC m=+15.611668445" Jul 06 14:24:40.085457 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.085417 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kfnjw" podStartSLOduration=3.642961224 podStartE2EDuration="15.085404164s" totalImagesPullingTime="11.44244294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:40.085008837 +0000 UTC m=+15.627696376" watchObservedRunningTime="2026-07-06 14:24:40.085404164 +0000 UTC m=+15.628091715" Jul 06 14:24:40.927278 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.927241 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:40.927459 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:40.927390 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:40.927459 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:40.927447 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:40.927591 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:40.927567 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:41.039945 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:41.039901 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzvkd" event={"ID":"3cb15cbe-aa6e-4e81-b5cd-ba0eade18021","Type":"ContainerStarted","Data":"4d8a75bebfd18d6d4e4a2f1921ab1fbf8a91a0f90e92c06a29e9742d11bc6c93"} Jul 06 14:24:41.059942 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:41.059880 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mzvkd" podStartSLOduration=4.611419377 podStartE2EDuration="16.059859692s" totalImagesPullingTime="11.448440315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:41.059369527 +0000 UTC m=+16.602057058" watchObservedRunningTime="2026-07-06 14:24:41.059859692 +0000 UTC m=+16.602547230" Jul 06 14:24:41.927075 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:41.927033 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:41.927281 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:41.927185 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:42.662929 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:42.662883 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:42.663317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:42.662945 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:42.663317 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.663055 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:42.663317 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.663080 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:42.663317 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.663127 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret podName:fcf896e5-cbff-43fa-8d73-b05d9be1a499 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:58.663106473 +0000 UTC m=+34.205794009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret") pod "global-pull-secret-syncer-rjl52" (UID: "fcf896e5-cbff-43fa-8d73-b05d9be1a499") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:42.663317 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.663153 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs podName:9b04e6a4-fe9b-4b78-954a-aa1927eb521a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:58.66314099 +0000 UTC m=+34.205828508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs") pod "network-metrics-daemon-wvgxw" (UID: "9b04e6a4-fe9b-4b78-954a-aa1927eb521a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:42.763735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:42.763695 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:42.763944 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.763905 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:24:42.763944 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.763931 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:24:42.763944 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.763944 2565 projected.go:196] Error preparing data for projected volume kube-api-access-s644p for pod openshift-network-diagnostics/network-check-target-9ls2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:42.764107 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.764015 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p podName:626f1048-dffc-4449-b748-ef625d7de486 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:58.763993676 +0000 UTC m=+34.306681196 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-s644p" (UniqueName: "kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p") pod "network-check-target-9ls2q" (UID: "626f1048-dffc-4449-b748-ef625d7de486") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:42.926448 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:42.926374 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:42.926636 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:42.926372 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:42.926636 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.926512 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:42.926636 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:42.926601 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:43.927496 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:43.927459 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:43.928196 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:43.927605 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:44.711539 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:44.711497 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:44.712166 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:44.712139 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:44.926946 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:44.926914 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:44.927113 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:44.926999 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:44.927113 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:44.927012 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:44.927233 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:44.927134 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:45.047979 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:45.047899 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:45.048560 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:45.048397 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-f9s5h" Jul 06 14:24:45.926421 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:45.926383 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:45.926590 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:45.926522 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:46.926501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:46.926466 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:46.927173 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:46.926466 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:46.927173 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:46.926600 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:46.927173 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:46.926677 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:47.926316 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:47.926278 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:47.926493 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:47.926409 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:48.238305 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:48.238102 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 14:24:48.893197 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:48.892709 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T14:24:48.238295379Z","UUID":"5c67639a-a832-4db3-8383-967938f34a83","Handler":null,"Name":"","Endpoint":""} Jul 06 14:24:48.894357 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:48.894335 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 14:24:48.894501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:48.894379 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 14:24:48.926371 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:48.926339 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:48.926561 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:48.926348 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:48.926561 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:48.926477 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:48.926561 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:48.926534 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:49.056127 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.056091 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7n788" event={"ID":"d7d6be71-c596-4932-a6a3-ffdacda9a326","Type":"ContainerStarted","Data":"a523fe5cb4a36de87b59144d9b7e16ab5f6cbefa02f063aa7bebc6dbe3964644"} Jul 06 14:24:49.057837 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.057815 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" event={"ID":"03b14c6a-25bf-470e-8495-4848f20e3728","Type":"ContainerStarted","Data":"aed328a5e235caca814b7a56ba8e6e51d569a0b200df2c7a8dc6369d6f390b51"} Jul 06 14:24:49.060475 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.060453 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"fcda7984d784b04ab0eb9d4e38f7f038967999a99dd3e42a65d81b0bba60a0b1"} Jul 06 14:24:49.060562 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.060509 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"259f3a4c35c45d0fb40140221a8c29c7c3f751fd0e50e98189bf5d1f97eea7c4"} Jul 06 14:24:49.060562 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.060538 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"5b21c6745655fd22c7daeeb8524dee2f68369b95d985a599cf50903da02a573c"} Jul 06 14:24:49.060632 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.060566 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"0ca3dbee22ed2f7e8a147038057d6e2e9ffa7bb9cbff2462d4e192ef235e4610"} Jul 06 14:24:49.060632 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.060603 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"01209a670072988025798a69b4eaf00397e7cfeecb186c75e26717263ea7bb6c"} Jul 06 14:24:49.060632 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.060630 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"4cc6d80aecbfa05689d0995ab6ea5bd07ed2535f963033ec0c225d6b28f12c06"} Jul 06 14:24:49.926841 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:49.926749 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:49.927217 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:49.926868 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:50.064673 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:50.064625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" event={"ID":"03b14c6a-25bf-470e-8495-4848f20e3728","Type":"ContainerStarted","Data":"46f7974bbdba77590f30e637b5334e96063f4ba49c907b9e6ce961a9d3d33106"} Jul 06 14:24:50.083731 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:50.083676 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-225hg" podStartSLOduration=3.27552821 podStartE2EDuration="25.083662095s" totalImagesPullingTime="21.808133885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:50.083223686 +0000 UTC m=+25.625911215" watchObservedRunningTime="2026-07-06 14:24:50.083662095 +0000 UTC m=+25.626349633" Jul 06 14:24:50.083943 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:50.083816 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7n788" podStartSLOduration=4.118961861 podStartE2EDuration="25.083807657s" totalImagesPullingTime="20.964845796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:49.07436865 +0000 UTC m=+24.617056190" watchObservedRunningTime="2026-07-06 14:24:50.083807657 +0000 UTC m=+25.626495189" Jul 06 14:24:50.926317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:50.926286 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:50.926317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:50.926297 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:50.926507 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:50.926398 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:50.926832 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:50.926776 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:51.070129 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:51.070054 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"13f7b4256db7da5197451ece8266843cf7f543592460a38f51073df0743cd02d"} Jul 06 14:24:51.926352 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:51.926314 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:51.926527 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:51.926429 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:52.927023 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:52.926984 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:52.927625 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:52.926984 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:52.927625 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:52.927124 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:52.927625 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:52.927204 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:53.926249 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:53.926197 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:53.926411 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:53.926340 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:54.927490 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:54.927242 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:54.927490 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:54.927484 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:54.928293 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:54.927315 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:54.928293 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:54.927548 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:55.079121 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.079083 2565 generic.go:356] "Generic (PLEG): container finished" podID="958ab8c5-dc0d-4a1e-a602-947c2a199009" containerID="3a778ff41cfb4d1af814fcac669caea25eab6348051276926ecf39e48a7be695" exitCode=0 Jul 06 14:24:55.079298 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.079170 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" event={"ID":"958ab8c5-dc0d-4a1e-a602-947c2a199009","Type":"ContainerDied","Data":"3a778ff41cfb4d1af814fcac669caea25eab6348051276926ecf39e48a7be695"} Jul 06 14:24:55.082919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.082891 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" event={"ID":"0c54d035-2286-4800-81cd-c71c8ff0e373","Type":"ContainerStarted","Data":"2a9e1e8b7723bd8c7faae3335451220c8b5fe23844174d5b9691387f495f43fd"} Jul 06 14:24:55.083231 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.083212 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:55.083339 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.083279 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:55.083339 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.083314 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:55.099104 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.099082 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:55.099204 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.099182 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:24:55.133670 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.133610 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" podStartSLOduration=9.138202378999999 podStartE2EDuration="30.133595117s" totalImagesPullingTime="20.995392738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:55.131728561 +0000 UTC m=+30.674416098" watchObservedRunningTime="2026-07-06 14:24:55.133595117 +0000 UTC m=+30.676282655" Jul 06 14:24:55.926258 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:55.926221 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:55.926393 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:55.926323 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:56.088003 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:56.087970 2565 generic.go:356] "Generic (PLEG): container finished" podID="958ab8c5-dc0d-4a1e-a602-947c2a199009" containerID="438b9737bc3a4d90e879827d335f290b7fb8e34a2c1bca189885408494a2fd43" exitCode=0 Jul 06 14:24:56.088377 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:56.088052 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" event={"ID":"958ab8c5-dc0d-4a1e-a602-947c2a199009","Type":"ContainerDied","Data":"438b9737bc3a4d90e879827d335f290b7fb8e34a2c1bca189885408494a2fd43"} Jul 06 14:24:56.185660 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:56.185590 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9ls2q"] Jul 06 14:24:56.185841 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:56.185735 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:56.185908 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:56.185884 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:56.186404 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:56.186376 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rjl52"] Jul 06 14:24:56.186524 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:56.186481 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:56.186621 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:56.186573 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:56.186973 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:56.186950 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wvgxw"] Jul 06 14:24:56.187062 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:56.187046 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:56.187208 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:56.187188 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:57.092891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:57.092858 2565 generic.go:356] "Generic (PLEG): container finished" podID="958ab8c5-dc0d-4a1e-a602-947c2a199009" containerID="b5a9f3e09b157c7394604d6eeb51639809c53cdcd7c9c6acd4d662a31945b9df" exitCode=0 Jul 06 14:24:57.093266 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:57.092945 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" event={"ID":"958ab8c5-dc0d-4a1e-a602-947c2a199009","Type":"ContainerDied","Data":"b5a9f3e09b157c7394604d6eeb51639809c53cdcd7c9c6acd4d662a31945b9df"} Jul 06 14:24:57.926682 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:57.926494 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:57.926887 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:57.926500 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:57.926887 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:57.926755 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:24:57.926887 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:57.926500 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:57.926887 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:57.926845 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:57.927016 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:57.926943 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:58.676240 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:58.676159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:58.676240 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:58.676196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:58.676632 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:58.676297 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:58.676632 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:58.676300 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:58.676632 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:58.676353 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs podName:9b04e6a4-fe9b-4b78-954a-aa1927eb521a nodeName:}" failed. No retries permitted until 2026-07-06 14:25:30.676336624 +0000 UTC m=+66.219024160 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs") pod "network-metrics-daemon-wvgxw" (UID: "9b04e6a4-fe9b-4b78-954a-aa1927eb521a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:24:58.676632 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:58.676366 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret podName:fcf896e5-cbff-43fa-8d73-b05d9be1a499 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:30.676360864 +0000 UTC m=+66.219048380 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret") pod "global-pull-secret-syncer-rjl52" (UID: "fcf896e5-cbff-43fa-8d73-b05d9be1a499") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:24:58.776645 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:58.776604 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:58.776817 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:58.776755 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:24:58.776817 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:58.776771 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:24:58.776817 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:58.776800 2565 projected.go:196] Error preparing data for projected volume kube-api-access-s644p for pod openshift-network-diagnostics/network-check-target-9ls2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:58.776920 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:58.776853 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p podName:626f1048-dffc-4449-b748-ef625d7de486 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:30.776840283 +0000 UTC m=+66.319527799 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-s644p" (UniqueName: "kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p") pod "network-check-target-9ls2q" (UID: "626f1048-dffc-4449-b748-ef625d7de486") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:24:59.926676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:59.926639 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:24:59.926676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:59.926693 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:24:59.927395 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:24:59.926774 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:24:59.927395 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:59.926802 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvgxw" podUID="9b04e6a4-fe9b-4b78-954a-aa1927eb521a" Jul 06 14:24:59.927395 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:59.926890 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9ls2q" podUID="626f1048-dffc-4449-b748-ef625d7de486" Jul 06 14:24:59.927395 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:24:59.926973 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rjl52" podUID="fcf896e5-cbff-43fa-8d73-b05d9be1a499" Jul 06 14:25:00.774294 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.774267 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-205.ec2.internal" event="NodeReady" Jul 06 14:25:00.774539 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.774423 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 14:25:00.818130 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.818047 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8psvj"] Jul 06 14:25:00.822221 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.822197 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fr7mf"] Jul 06 14:25:00.822386 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.822349 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.825068 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.825045 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fr7mf" Jul 06 14:25:00.825284 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.825262 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 14:25:00.825393 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.825292 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 14:25:00.825479 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.825452 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s54bc\"" Jul 06 14:25:00.828647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.828051 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 14:25:00.828647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.828285 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 14:25:00.828647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.828534 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xqgrr\"" Jul 06 14:25:00.829129 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.829111 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 14:25:00.833038 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.833017 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8psvj"] Jul 06 14:25:00.835480 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.835456 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fr7mf"] Jul 06 14:25:00.892351 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.892316 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a04e7fb5-c4dc-48be-9748-428d658f9ca1-config-volume\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.892532 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.892355 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.892532 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.892428 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4qfr\" (UniqueName: \"kubernetes.io/projected/a04e7fb5-c4dc-48be-9748-428d658f9ca1-kube-api-access-n4qfr\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.892532 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.892477 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a04e7fb5-c4dc-48be-9748-428d658f9ca1-tmp-dir\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.993008 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.992978 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abb581bd-675d-43c4-9e11-da3095e4109d-cert\") pod \"ingress-canary-fr7mf\" (UID: \"abb581bd-675d-43c4-9e11-da3095e4109d\") " pod="openshift-ingress-canary/ingress-canary-fr7mf" Jul 06 14:25:00.993598 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.993025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a04e7fb5-c4dc-48be-9748-428d658f9ca1-config-volume\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.993598 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.993088 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.993598 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.993121 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g6br\" (UniqueName: \"kubernetes.io/projected/abb581bd-675d-43c4-9e11-da3095e4109d-kube-api-access-9g6br\") pod \"ingress-canary-fr7mf\" (UID: \"abb581bd-675d-43c4-9e11-da3095e4109d\") " pod="openshift-ingress-canary/ingress-canary-fr7mf" Jul 06 14:25:00.993598 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.993160 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4qfr\" (UniqueName: \"kubernetes.io/projected/a04e7fb5-c4dc-48be-9748-428d658f9ca1-kube-api-access-n4qfr\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.993598 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.993190 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a04e7fb5-c4dc-48be-9748-428d658f9ca1-tmp-dir\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.993598 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:00.993223 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:25:00.993900 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:00.993696 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls podName:a04e7fb5-c4dc-48be-9748-428d658f9ca1 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:01.49366658 +0000 UTC m=+37.036354116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls") pod "dns-default-8psvj" (UID: "a04e7fb5-c4dc-48be-9748-428d658f9ca1") : secret "dns-default-metrics-tls" not found Jul 06 14:25:00.993900 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.993852 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a04e7fb5-c4dc-48be-9748-428d658f9ca1-tmp-dir\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:00.994074 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:00.993955 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a04e7fb5-c4dc-48be-9748-428d658f9ca1-config-volume\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:01.006072 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.006039 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4qfr\" (UniqueName: \"kubernetes.io/projected/a04e7fb5-c4dc-48be-9748-428d658f9ca1-kube-api-access-n4qfr\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:01.094176 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.094089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abb581bd-675d-43c4-9e11-da3095e4109d-cert\") pod \"ingress-canary-fr7mf\" (UID: \"abb581bd-675d-43c4-9e11-da3095e4109d\") " pod="openshift-ingress-canary/ingress-canary-fr7mf" Jul 06 14:25:01.094176 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.094156 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6br\" (UniqueName: \"kubernetes.io/projected/abb581bd-675d-43c4-9e11-da3095e4109d-kube-api-access-9g6br\") pod \"ingress-canary-fr7mf\" (UID: \"abb581bd-675d-43c4-9e11-da3095e4109d\") " pod="openshift-ingress-canary/ingress-canary-fr7mf" Jul 06 14:25:01.097451 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.097417 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abb581bd-675d-43c4-9e11-da3095e4109d-cert\") pod \"ingress-canary-fr7mf\" (UID: \"abb581bd-675d-43c4-9e11-da3095e4109d\") " pod="openshift-ingress-canary/ingress-canary-fr7mf" Jul 06 14:25:01.104727 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.104700 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6br\" (UniqueName: \"kubernetes.io/projected/abb581bd-675d-43c4-9e11-da3095e4109d-kube-api-access-9g6br\") pod \"ingress-canary-fr7mf\" (UID: \"abb581bd-675d-43c4-9e11-da3095e4109d\") " pod="openshift-ingress-canary/ingress-canary-fr7mf" Jul 06 14:25:01.141676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.141632 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fr7mf" Jul 06 14:25:01.498049 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.498010 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:01.498243 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:01.498159 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:25:01.498243 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:01.498235 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls podName:a04e7fb5-c4dc-48be-9748-428d658f9ca1 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:02.498214533 +0000 UTC m=+38.040902049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls") pod "dns-default-8psvj" (UID: "a04e7fb5-c4dc-48be-9748-428d658f9ca1") : secret "dns-default-metrics-tls" not found Jul 06 14:25:01.926977 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.926890 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:25:01.927143 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.926987 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:25:01.927143 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.927077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:25:01.930080 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.930058 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:25:01.932004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.931639 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 14:25:01.932004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.931680 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-689l7\"" Jul 06 14:25:01.932004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.931730 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 14:25:01.932004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.931742 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fzm8z\"" Jul 06 14:25:01.932004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:01.931685 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:25:02.506421 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:02.506236 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:02.507043 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:02.506388 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:25:02.507043 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:02.506569 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls podName:a04e7fb5-c4dc-48be-9748-428d658f9ca1 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:04.506545294 +0000 UTC m=+40.049232825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls") pod "dns-default-8psvj" (UID: "a04e7fb5-c4dc-48be-9748-428d658f9ca1") : secret "dns-default-metrics-tls" not found Jul 06 14:25:02.525176 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:02.525145 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fr7mf"] Jul 06 14:25:02.606107 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:02.606079 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb581bd_675d_43c4_9e11_da3095e4109d.slice/crio-bb8e1ac1d85fa499f8df094156014a779962e65fd3c799b0f7cff2a4f7a8e278 WatchSource:0}: Error finding container bb8e1ac1d85fa499f8df094156014a779962e65fd3c799b0f7cff2a4f7a8e278: Status 404 returned error can't find the container with id bb8e1ac1d85fa499f8df094156014a779962e65fd3c799b0f7cff2a4f7a8e278 Jul 06 14:25:03.109726 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:03.109693 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fr7mf" event={"ID":"abb581bd-675d-43c4-9e11-da3095e4109d","Type":"ContainerStarted","Data":"bb8e1ac1d85fa499f8df094156014a779962e65fd3c799b0f7cff2a4f7a8e278"} Jul 06 14:25:03.112562 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:03.112535 2565 generic.go:356] "Generic (PLEG): container finished" podID="958ab8c5-dc0d-4a1e-a602-947c2a199009" containerID="93bf6b414fce6bdccaf2ea09e1e111311ea3c1eaf389628aff1e3963b9817204" exitCode=0 Jul 06 14:25:03.112692 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:03.112570 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" event={"ID":"958ab8c5-dc0d-4a1e-a602-947c2a199009","Type":"ContainerDied","Data":"93bf6b414fce6bdccaf2ea09e1e111311ea3c1eaf389628aff1e3963b9817204"} Jul 06 14:25:04.118413 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:04.118378 2565 generic.go:356] "Generic (PLEG): container finished" podID="958ab8c5-dc0d-4a1e-a602-947c2a199009" containerID="d47535ca7deaa0cb8d31ff0a8dc0458194113208518cc37ebab6d2b978966f99" exitCode=0 Jul 06 14:25:04.118845 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:04.118434 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" event={"ID":"958ab8c5-dc0d-4a1e-a602-947c2a199009","Type":"ContainerDied","Data":"d47535ca7deaa0cb8d31ff0a8dc0458194113208518cc37ebab6d2b978966f99"} Jul 06 14:25:04.521858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:04.521810 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:04.522029 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:04.521936 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:25:04.522029 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:04.521993 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls podName:a04e7fb5-c4dc-48be-9748-428d658f9ca1 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:08.52197807 +0000 UTC m=+44.064665590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls") pod "dns-default-8psvj" (UID: "a04e7fb5-c4dc-48be-9748-428d658f9ca1") : secret "dns-default-metrics-tls" not found Jul 06 14:25:05.125700 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:05.125663 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" event={"ID":"958ab8c5-dc0d-4a1e-a602-947c2a199009","Type":"ContainerStarted","Data":"103c5722cf991db3f412b1e32136c1d585195ec7e636122639e3e49a76953883"} Jul 06 14:25:05.152569 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:05.152508 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nd4wx" podStartSLOduration=6.943639765 podStartE2EDuration="40.152488575s" totalImagesPullingTime="33.20884881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:05.150910079 +0000 UTC m=+40.693597617" watchObservedRunningTime="2026-07-06 14:25:05.152488575 +0000 UTC m=+40.695176114" Jul 06 14:25:07.131642 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:07.131461 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fr7mf" event={"ID":"abb581bd-675d-43c4-9e11-da3095e4109d","Type":"ContainerStarted","Data":"12adee121f514d8714009010925a84a7d76f22ec139f14fde678fbbd45221ded"} Jul 06 14:25:07.160491 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:07.160442 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fr7mf" podStartSLOduration=3.633274959 podStartE2EDuration="7.160426435s" totalImagesPullingTime="3.527151476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:07.160072713 +0000 UTC m=+42.702760251" watchObservedRunningTime="2026-07-06 14:25:07.160426435 +0000 UTC m=+42.703113973" Jul 06 14:25:08.551587 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:08.551540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:08.552008 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:08.551666 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:25:08.552008 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:08.551719 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls podName:a04e7fb5-c4dc-48be-9748-428d658f9ca1 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:16.551706241 +0000 UTC m=+52.094393760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls") pod "dns-default-8psvj" (UID: "a04e7fb5-c4dc-48be-9748-428d658f9ca1") : secret "dns-default-metrics-tls" not found Jul 06 14:25:10.190184 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.190153 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr"] Jul 06 14:25:10.220407 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.220370 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr"] Jul 06 14:25:10.220407 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.220411 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5477b4dcdf-nnrp4"] Jul 06 14:25:10.220615 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.220490 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr" Jul 06 14:25:10.223550 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.223530 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-skm46\"" Jul 06 14:25:10.223685 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.223547 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:25:10.223685 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.223622 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 14:25:10.244367 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.244342 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5477b4dcdf-nnrp4"] Jul 06 14:25:10.244508 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.244436 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.247420 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.247392 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 14:25:10.247561 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.247461 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-m9tb9\"" Jul 06 14:25:10.247561 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.247483 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 14:25:10.247561 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.247501 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 14:25:10.247805 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.247773 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 14:25:10.247867 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.247775 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 14:25:10.247867 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.247827 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 14:25:10.287916 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.287881 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-585f7947c-llrbp"] Jul 06 14:25:10.306383 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.306347 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p"] Jul 06 14:25:10.306520 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.306479 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.309381 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.309350 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 14:25:10.309523 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.309446 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 14:25:10.309617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.309599 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 14:25:10.309674 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.309617 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9dzg8\"" Jul 06 14:25:10.315040 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.315013 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 14:25:10.322489 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.322463 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-585f7947c-llrbp"] Jul 06 14:25:10.322489 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.322494 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p"] Jul 06 14:25:10.322652 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.322584 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.325280 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.325259 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:25:10.325400 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.325258 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 14:25:10.325400 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.325335 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 14:25:10.325400 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.325261 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-nqn9f\"" Jul 06 14:25:10.325544 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.325480 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 14:25:10.364157 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.364117 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.364333 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.364175 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-stats-auth\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.364333 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.364252 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-default-certificate\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.364333 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.364286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.364333 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.364315 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wdlg\" (UniqueName: \"kubernetes.io/projected/5e221b5b-b395-4184-9dfd-709037b5d5fd-kube-api-access-5wdlg\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.364502 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.364349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lqcr\" (UniqueName: \"kubernetes.io/projected/75ec02f5-9d55-4fea-b1ab-f8ec18d7151a-kube-api-access-5lqcr\") pod \"volume-data-source-validator-5d456fdbcd-2lqbr\" (UID: \"75ec02f5-9d55-4fea-b1ab-f8ec18d7151a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr" Jul 06 14:25:10.391366 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.391332 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j"] Jul 06 14:25:10.416568 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.416534 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j"] Jul 06 14:25:10.416762 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.416658 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:10.419402 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.419380 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 14:25:10.419511 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.419428 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:25:10.419511 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.419487 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 14:25:10.419748 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.419732 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tdk8s\"" Jul 06 14:25:10.465422 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465342 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wdlg\" (UniqueName: \"kubernetes.io/projected/5e221b5b-b395-4184-9dfd-709037b5d5fd-kube-api-access-5wdlg\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.465422 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465373 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-trusted-ca\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.465422 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c63ad8-e362-4adc-bfc7-c3b688665c8f-ca-trust-extracted\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.465647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465457 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lqcr\" (UniqueName: \"kubernetes.io/projected/75ec02f5-9d55-4fea-b1ab-f8ec18d7151a-kube-api-access-5lqcr\") pod \"volume-data-source-validator-5d456fdbcd-2lqbr\" (UID: \"75ec02f5-9d55-4fea-b1ab-f8ec18d7151a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr" Jul 06 14:25:10.465647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465490 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-bound-sa-token\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.465647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465542 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.465647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465561 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-installation-pull-secrets\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.465647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbvld\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-kube-api-access-kbvld\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.465647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465603 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-stats-auth\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.465894 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.465650 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:25:10.465894 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.465703 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:10.965683452 +0000 UTC m=+46.508370968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : secret "router-metrics-certs-default" not found Jul 06 14:25:10.465894 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7f9c177-4e04-4842-9c10-828e0cc83929-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.465894 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465779 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-image-registry-private-configuration\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.465894 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-default-certificate\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.465894 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465879 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.466099 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.466099 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465934 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vdvx\" (UniqueName: \"kubernetes.io/projected/d7f9c177-4e04-4842-9c10-828e0cc83929-kube-api-access-7vdvx\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.466099 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465961 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-certificates\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.466099 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.465988 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7f9c177-4e04-4842-9c10-828e0cc83929-config\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.466099 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.466017 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:10.965998827 +0000 UTC m=+46.508686362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : configmap references non-existent config key: service-ca.crt Jul 06 14:25:10.469862 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.469836 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-stats-auth\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.469967 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.469902 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-default-certificate\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.482359 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.482337 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lqcr\" (UniqueName: \"kubernetes.io/projected/75ec02f5-9d55-4fea-b1ab-f8ec18d7151a-kube-api-access-5lqcr\") pod \"volume-data-source-validator-5d456fdbcd-2lqbr\" (UID: \"75ec02f5-9d55-4fea-b1ab-f8ec18d7151a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr" Jul 06 14:25:10.492803 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.492763 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wdlg\" (UniqueName: \"kubernetes.io/projected/5e221b5b-b395-4184-9dfd-709037b5d5fd-kube-api-access-5wdlg\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.529605 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.529566 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr" Jul 06 14:25:10.567064 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567037 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-installation-pull-secrets\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.567064 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567068 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbvld\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-kube-api-access-kbvld\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.567271 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567094 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7f9c177-4e04-4842-9c10-828e0cc83929-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.567271 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567124 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-image-registry-private-configuration\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.567271 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:10.567271 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567186 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.567271 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567223 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vdvx\" (UniqueName: \"kubernetes.io/projected/d7f9c177-4e04-4842-9c10-828e0cc83929-kube-api-access-7vdvx\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.567495 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567283 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-certificates\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.567495 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.567314 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:25:10.567495 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.567329 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-585f7947c-llrbp: secret "image-registry-tls" not found Jul 06 14:25:10.567495 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567331 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7f9c177-4e04-4842-9c10-828e0cc83929-config\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.567495 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567359 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-trusted-ca\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.567495 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.567395 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls podName:64c63ad8-e362-4adc-bfc7-c3b688665c8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:11.067374092 +0000 UTC m=+46.610061608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls") pod "image-registry-585f7947c-llrbp" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f") : secret "image-registry-tls" not found Jul 06 14:25:10.567495 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567414 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdzlx\" (UniqueName: \"kubernetes.io/projected/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-kube-api-access-xdzlx\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:10.567495 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567466 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c63ad8-e362-4adc-bfc7-c3b688665c8f-ca-trust-extracted\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.567916 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.567502 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-bound-sa-token\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.568347 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.568292 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-certificates\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.568422 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.568346 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7f9c177-4e04-4842-9c10-828e0cc83929-config\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.568481 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.568455 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-trusted-ca\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.568775 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.568758 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c63ad8-e362-4adc-bfc7-c3b688665c8f-ca-trust-extracted\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.569907 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.569886 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-image-registry-private-configuration\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.570001 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.569944 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-installation-pull-secrets\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.570001 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.569965 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7f9c177-4e04-4842-9c10-828e0cc83929-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.580315 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.580285 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-bound-sa-token\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.580518 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.580502 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vdvx\" (UniqueName: \"kubernetes.io/projected/d7f9c177-4e04-4842-9c10-828e0cc83929-kube-api-access-7vdvx\") pod \"service-ca-operator-5d46d9ffd4-ss27p\" (UID: \"d7f9c177-4e04-4842-9c10-828e0cc83929\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.581195 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.581173 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbvld\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-kube-api-access-kbvld\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:10.631069 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.631040 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" Jul 06 14:25:10.665355 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.665330 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr"] Jul 06 14:25:10.668600 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.668574 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:10.668699 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.668658 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdzlx\" (UniqueName: \"kubernetes.io/projected/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-kube-api-access-xdzlx\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:10.668699 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:10.668580 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ec02f5_9d55_4fea_b1ab_f8ec18d7151a.slice/crio-a9264bf2d8a000bb222e177384e84e831345790c65c321b5472082f8c7c80a5d WatchSource:0}: Error finding container a9264bf2d8a000bb222e177384e84e831345790c65c321b5472082f8c7c80a5d: Status 404 returned error can't find the container with id a9264bf2d8a000bb222e177384e84e831345790c65c321b5472082f8c7c80a5d Jul 06 14:25:10.668699 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.668664 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:25:10.668863 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.668723 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls podName:018655ad-3ef8-43f2-a2b5-cdb7c5718d5e nodeName:}" failed. No retries permitted until 2026-07-06 14:25:11.168709623 +0000 UTC m=+46.711397152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-skb8j" (UID: "018655ad-3ef8-43f2-a2b5-cdb7c5718d5e") : secret "samples-operator-tls" not found Jul 06 14:25:10.678085 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.678063 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdzlx\" (UniqueName: \"kubernetes.io/projected/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-kube-api-access-xdzlx\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:10.783044 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.782880 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p"] Jul 06 14:25:10.785252 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:10.785226 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f9c177_4e04_4842_9c10_828e0cc83929.slice/crio-eb92591cf1f2fda9dfdb802789fbe6cb0d513f8de5a9c8aa72ade49db7c4bc0d WatchSource:0}: Error finding container eb92591cf1f2fda9dfdb802789fbe6cb0d513f8de5a9c8aa72ade49db7c4bc0d: Status 404 returned error can't find the container with id eb92591cf1f2fda9dfdb802789fbe6cb0d513f8de5a9c8aa72ade49db7c4bc0d Jul 06 14:25:10.971528 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.971498 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.971708 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:10.971580 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:10.971708 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.971676 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:11.971657775 +0000 UTC m=+47.514345294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : configmap references non-existent config key: service-ca.crt Jul 06 14:25:10.971708 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.971683 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:25:10.971849 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:10.971774 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:11.97176005 +0000 UTC m=+47.514447574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : secret "router-metrics-certs-default" not found Jul 06 14:25:11.072748 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:11.072650 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:11.072900 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:11.072819 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:25:11.072900 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:11.072837 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-585f7947c-llrbp: secret "image-registry-tls" not found Jul 06 14:25:11.072900 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:11.072896 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls podName:64c63ad8-e362-4adc-bfc7-c3b688665c8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:12.072878171 +0000 UTC m=+47.615565706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls") pod "image-registry-585f7947c-llrbp" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f") : secret "image-registry-tls" not found Jul 06 14:25:11.141854 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:11.141817 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" event={"ID":"d7f9c177-4e04-4842-9c10-828e0cc83929","Type":"ContainerStarted","Data":"eb92591cf1f2fda9dfdb802789fbe6cb0d513f8de5a9c8aa72ade49db7c4bc0d"} Jul 06 14:25:11.142980 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:11.142955 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr" event={"ID":"75ec02f5-9d55-4fea-b1ab-f8ec18d7151a","Type":"ContainerStarted","Data":"a9264bf2d8a000bb222e177384e84e831345790c65c321b5472082f8c7c80a5d"} Jul 06 14:25:11.174054 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:11.174020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:11.174190 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:11.174128 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:25:11.174190 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:11.174179 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls podName:018655ad-3ef8-43f2-a2b5-cdb7c5718d5e nodeName:}" failed. No retries permitted until 2026-07-06 14:25:12.174165499 +0000 UTC m=+47.716853018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-skb8j" (UID: "018655ad-3ef8-43f2-a2b5-cdb7c5718d5e") : secret "samples-operator-tls" not found Jul 06 14:25:11.980973 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:11.980931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:11.981867 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:11.981056 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:11.981867 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:11.981110 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:25:11.981867 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:11.981186 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:13.981165375 +0000 UTC m=+49.523852908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : secret "router-metrics-certs-default" not found Jul 06 14:25:11.981867 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:11.981237 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:13.981215804 +0000 UTC m=+49.523903337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : configmap references non-existent config key: service-ca.crt Jul 06 14:25:12.081936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:12.081899 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:12.082107 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:12.082035 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:25:12.082107 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:12.082052 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-585f7947c-llrbp: secret "image-registry-tls" not found Jul 06 14:25:12.082219 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:12.082115 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls podName:64c63ad8-e362-4adc-bfc7-c3b688665c8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:14.082096973 +0000 UTC m=+49.624784501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls") pod "image-registry-585f7947c-llrbp" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f") : secret "image-registry-tls" not found Jul 06 14:25:12.182849 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:12.182810 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:12.183030 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:12.183002 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:25:12.183098 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:12.183082 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls podName:018655ad-3ef8-43f2-a2b5-cdb7c5718d5e nodeName:}" failed. No retries permitted until 2026-07-06 14:25:14.183060948 +0000 UTC m=+49.725748485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-skb8j" (UID: "018655ad-3ef8-43f2-a2b5-cdb7c5718d5e") : secret "samples-operator-tls" not found Jul 06 14:25:13.999330 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:13.999227 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:13.999330 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:13.999324 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:13.999856 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:13.999419 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:25:13.999856 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:13.999473 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:17.99945164 +0000 UTC m=+53.542139169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : configmap references non-existent config key: service-ca.crt Jul 06 14:25:13.999856 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:13.999495 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:17.999485687 +0000 UTC m=+53.542173219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : secret "router-metrics-certs-default" not found Jul 06 14:25:14.099937 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:14.099894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:14.100107 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:14.100050 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:25:14.100107 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:14.100081 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-585f7947c-llrbp: secret "image-registry-tls" not found Jul 06 14:25:14.100237 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:14.100153 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls podName:64c63ad8-e362-4adc-bfc7-c3b688665c8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:18.100131568 +0000 UTC m=+53.642819101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls") pod "image-registry-585f7947c-llrbp" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f") : secret "image-registry-tls" not found Jul 06 14:25:14.156143 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:14.156109 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" event={"ID":"d7f9c177-4e04-4842-9c10-828e0cc83929","Type":"ContainerStarted","Data":"3346074c9e3e91bde0d81cd34c9ce51e07190b0697b212f0b861baeb319425a2"} Jul 06 14:25:14.157724 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:14.157702 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr" event={"ID":"75ec02f5-9d55-4fea-b1ab-f8ec18d7151a","Type":"ContainerStarted","Data":"24e8299df0626d4c63cb6b172e46d2af2a668be03d41028eb1772ed538e979e4"} Jul 06 14:25:14.173046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:14.172999 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" podStartSLOduration=1.336364681 podStartE2EDuration="4.172985821s" totalImagesPullingTime="2.83662114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:14.171911976 +0000 UTC m=+49.714599516" watchObservedRunningTime="2026-07-06 14:25:14.172985821 +0000 UTC m=+49.715673358" Jul 06 14:25:14.188918 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:14.188865 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2lqbr" podStartSLOduration=1.979982232 podStartE2EDuration="4.188845272s" totalImagesPullingTime="2.20886304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:14.188095772 +0000 UTC m=+49.730783314" watchObservedRunningTime="2026-07-06 14:25:14.188845272 +0000 UTC m=+49.731532809" Jul 06 14:25:14.201185 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:14.201158 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:14.201452 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:14.201342 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:25:14.201452 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:14.201428 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls podName:018655ad-3ef8-43f2-a2b5-cdb7c5718d5e nodeName:}" failed. No retries permitted until 2026-07-06 14:25:18.201408547 +0000 UTC m=+53.744096082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-skb8j" (UID: "018655ad-3ef8-43f2-a2b5-cdb7c5718d5e") : secret "samples-operator-tls" not found Jul 06 14:25:16.621538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:16.621495 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:16.622030 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:16.621622 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:25:16.622030 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:16.621681 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls podName:a04e7fb5-c4dc-48be-9748-428d658f9ca1 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:32.62166394 +0000 UTC m=+68.164351459 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls") pod "dns-default-8psvj" (UID: "a04e7fb5-c4dc-48be-9748-428d658f9ca1") : secret "dns-default-metrics-tls" not found Jul 06 14:25:17.076195 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.076153 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-whrlv"] Jul 06 14:25:17.101545 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.101516 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.102025 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.102000 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-whrlv"] Jul 06 14:25:17.106400 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.106380 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 14:25:17.107745 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.107728 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 14:25:17.107892 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.107862 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 14:25:17.108045 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.108021 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mh8w\"" Jul 06 14:25:17.108106 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.107983 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 14:25:17.125925 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.125893 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.125925 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.125933 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.126109 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.126021 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-data-volume\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.126109 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.126061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxwv2\" (UniqueName: \"kubernetes.io/projected/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-api-access-sxwv2\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.126172 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.126136 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-crio-socket\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.227040 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.226998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-crio-socket\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.227209 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.227082 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.227209 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.227105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.227209 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.227177 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-data-volume\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.227355 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.227217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxwv2\" (UniqueName: \"kubernetes.io/projected/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-api-access-sxwv2\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.227355 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:17.227239 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:25:17.227355 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.227262 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-crio-socket\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.227355 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:17.227279 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:25:17.227355 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:17.227299 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:17.727284866 +0000 UTC m=+53.269972382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:25:17.227355 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:17.227342 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:17.727328498 +0000 UTC m=+53.270016018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : configmap "kube-rbac-proxy" not found Jul 06 14:25:17.227643 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.227583 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-data-volume\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.239842 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.239810 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxwv2\" (UniqueName: \"kubernetes.io/projected/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-api-access-sxwv2\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.733470 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.732816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.733470 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.732909 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:17.733470 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:17.732931 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:25:17.733470 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:17.733093 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:18.733067013 +0000 UTC m=+54.275754553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:25:17.733470 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:17.733136 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:25:17.734147 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:17.733571 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:18.733530562 +0000 UTC m=+54.276218095 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : configmap "kube-rbac-proxy" not found Jul 06 14:25:17.752242 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.752210 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-c58jw"] Jul 06 14:25:17.774574 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.774540 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-c58jw"] Jul 06 14:25:17.774712 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.774644 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.777469 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.777441 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 14:25:17.777593 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.777498 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 14:25:17.778897 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.778878 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 14:25:17.779001 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.778913 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-67ssr\"" Jul 06 14:25:17.779001 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.778880 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 14:25:17.834153 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.834120 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxqk8\" (UniqueName: \"kubernetes.io/projected/7c201b93-0155-44ff-bb5a-9406a1cc99aa-kube-api-access-zxqk8\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.834292 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.834162 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7c201b93-0155-44ff-bb5a-9406a1cc99aa-signing-cabundle\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.834292 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.834196 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7c201b93-0155-44ff-bb5a-9406a1cc99aa-signing-key\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.935316 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.935278 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxqk8\" (UniqueName: \"kubernetes.io/projected/7c201b93-0155-44ff-bb5a-9406a1cc99aa-kube-api-access-zxqk8\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.935316 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.935320 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7c201b93-0155-44ff-bb5a-9406a1cc99aa-signing-cabundle\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.935519 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.935356 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7c201b93-0155-44ff-bb5a-9406a1cc99aa-signing-key\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.936024 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.935994 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7c201b93-0155-44ff-bb5a-9406a1cc99aa-signing-cabundle\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.937844 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.937823 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7c201b93-0155-44ff-bb5a-9406a1cc99aa-signing-key\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:17.948855 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:17.948829 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxqk8\" (UniqueName: \"kubernetes.io/projected/7c201b93-0155-44ff-bb5a-9406a1cc99aa-kube-api-access-zxqk8\") pod \"service-ca-758f599c5-c58jw\" (UID: \"7c201b93-0155-44ff-bb5a-9406a1cc99aa\") " pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:18.036221 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:18.036138 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:18.036221 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:18.036217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:18.036396 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.036297 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:25:18.036396 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.036357 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:26.036339774 +0000 UTC m=+61.579027291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : configmap references non-existent config key: service-ca.crt Jul 06 14:25:18.036396 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.036382 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:26.036376038 +0000 UTC m=+61.579063554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : secret "router-metrics-certs-default" not found Jul 06 14:25:18.083089 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:18.083062 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-c58jw" Jul 06 14:25:18.137199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:18.137162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:18.137350 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.137325 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:25:18.137350 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.137347 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-585f7947c-llrbp: secret "image-registry-tls" not found Jul 06 14:25:18.137420 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.137400 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls podName:64c63ad8-e362-4adc-bfc7-c3b688665c8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:26.137384082 +0000 UTC m=+61.680071598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls") pod "image-registry-585f7947c-llrbp" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f") : secret "image-registry-tls" not found Jul 06 14:25:18.234947 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:18.234917 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-c58jw"] Jul 06 14:25:18.238104 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:18.238082 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:18.238221 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:18.238198 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c201b93_0155_44ff_bb5a_9406a1cc99aa.slice/crio-6d20e6fd57738176a8be61942a641da5b5415fb068b5b9bda68578db5ada3c35 WatchSource:0}: Error finding container 6d20e6fd57738176a8be61942a641da5b5415fb068b5b9bda68578db5ada3c35: Status 404 returned error can't find the container with id 6d20e6fd57738176a8be61942a641da5b5415fb068b5b9bda68578db5ada3c35 Jul 06 14:25:18.238268 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.238232 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:25:18.238301 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.238296 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls podName:018655ad-3ef8-43f2-a2b5-cdb7c5718d5e nodeName:}" failed. No retries permitted until 2026-07-06 14:25:26.238280718 +0000 UTC m=+61.780968253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-skb8j" (UID: "018655ad-3ef8-43f2-a2b5-cdb7c5718d5e") : secret "samples-operator-tls" not found Jul 06 14:25:18.742910 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:18.742867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:18.742910 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:18.742911 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:18.743411 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.743024 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:25:18.743411 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.743034 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:25:18.743411 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.743089 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:20.743069691 +0000 UTC m=+56.285757209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : configmap "kube-rbac-proxy" not found Jul 06 14:25:18.743411 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:18.743191 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:20.743179466 +0000 UTC m=+56.285866982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:25:19.171850 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:19.171757 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-c58jw" event={"ID":"7c201b93-0155-44ff-bb5a-9406a1cc99aa","Type":"ContainerStarted","Data":"c3dfe63cd2ac08ba5dfdcc346d9af2f97a209fc9c3a34a2b62a83cfe6b8be63c"} Jul 06 14:25:19.171850 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:19.171826 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-c58jw" event={"ID":"7c201b93-0155-44ff-bb5a-9406a1cc99aa","Type":"ContainerStarted","Data":"6d20e6fd57738176a8be61942a641da5b5415fb068b5b9bda68578db5ada3c35"} Jul 06 14:25:19.190322 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:19.190267 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-c58jw" podStartSLOduration=2.190247744 podStartE2EDuration="2.190247744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:19.18912106 +0000 UTC m=+54.731808600" watchObservedRunningTime="2026-07-06 14:25:19.190247744 +0000 UTC m=+54.732935283" Jul 06 14:25:19.269028 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:19.269001 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5rtnh_3e3a655e-a83a-43a2-b610-6b66bf7c6fc0/dns-node-resolver/0.log" Jul 06 14:25:20.466778 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:20.466750 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kfnjw_8ca6503a-77c2-47ef-8587-531b6ee00e34/node-ca/0.log" Jul 06 14:25:20.760178 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:20.760077 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:20.760178 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:20.760124 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:20.760406 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:20.760249 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:25:20.760406 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:20.760268 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:25:20.760406 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:20.760306 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:24.760288704 +0000 UTC m=+60.302976226 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : configmap "kube-rbac-proxy" not found Jul 06 14:25:20.760406 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:20.760345 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:24.760327016 +0000 UTC m=+60.303014547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:25:21.066623 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:21.066538 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fr7mf_abb581bd-675d-43c4-9e11-da3095e4109d/serve-healthcheck-canary/0.log" Jul 06 14:25:24.796701 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:24.796664 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:24.797075 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:24.796836 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:25:24.797075 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:24.796864 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:24.797075 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:24.796906 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:32.796885111 +0000 UTC m=+68.339572638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : configmap "kube-rbac-proxy" not found Jul 06 14:25:24.797075 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:24.796962 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:25:24.797075 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:24.797005 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls podName:8f0a0a51-50a8-4499-b7dd-c2445bb41b8f nodeName:}" failed. No retries permitted until 2026-07-06 14:25:32.796993364 +0000 UTC m=+68.339680886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-whrlv" (UID: "8f0a0a51-50a8-4499-b7dd-c2445bb41b8f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:25:26.107894 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.107844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:26.108358 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.107964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:26.108358 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:26.108030 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle podName:5e221b5b-b395-4184-9dfd-709037b5d5fd nodeName:}" failed. No retries permitted until 2026-07-06 14:25:42.108011861 +0000 UTC m=+77.650699380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle") pod "router-default-5477b4dcdf-nnrp4" (UID: "5e221b5b-b395-4184-9dfd-709037b5d5fd") : configmap references non-existent config key: service-ca.crt Jul 06 14:25:26.110408 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.110389 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e221b5b-b395-4184-9dfd-709037b5d5fd-metrics-certs\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:26.208871 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.208837 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:26.211487 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.211462 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"image-registry-585f7947c-llrbp\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:26.220707 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.220687 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9dzg8\"" Jul 06 14:25:26.229046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.229023 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:26.310111 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.310065 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:26.313317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.313284 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/018655ad-3ef8-43f2-a2b5-cdb7c5718d5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-skb8j\" (UID: \"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:26.329148 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.329100 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tdk8s\"" Jul 06 14:25:26.336022 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.335998 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" Jul 06 14:25:26.393464 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.393431 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-585f7947c-llrbp"] Jul 06 14:25:26.400250 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:26.400222 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c63ad8_e362_4adc_bfc7_c3b688665c8f.slice/crio-7019326a695ee21514abec7d8130966c2b4fdc950dd530f2e1729e36a09f5f44 WatchSource:0}: Error finding container 7019326a695ee21514abec7d8130966c2b4fdc950dd530f2e1729e36a09f5f44: Status 404 returned error can't find the container with id 7019326a695ee21514abec7d8130966c2b4fdc950dd530f2e1729e36a09f5f44 Jul 06 14:25:26.499900 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:26.499855 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j"] Jul 06 14:25:27.103539 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:27.103516 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rw6gh" Jul 06 14:25:27.197879 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:27.197837 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" event={"ID":"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e","Type":"ContainerStarted","Data":"fc9bb8445e216bbc6cc68337838da248a0065319090b685f6de0153fcf5f765e"} Jul 06 14:25:27.199853 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:27.199825 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-585f7947c-llrbp" event={"ID":"64c63ad8-e362-4adc-bfc7-c3b688665c8f","Type":"ContainerStarted","Data":"552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e"} Jul 06 14:25:27.200002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:27.199871 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-585f7947c-llrbp" event={"ID":"64c63ad8-e362-4adc-bfc7-c3b688665c8f","Type":"ContainerStarted","Data":"7019326a695ee21514abec7d8130966c2b4fdc950dd530f2e1729e36a09f5f44"} Jul 06 14:25:27.200062 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:27.199994 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:27.221215 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:27.221171 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-585f7947c-llrbp" podStartSLOduration=17.221157326 podStartE2EDuration="17.221157326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:27.219532637 +0000 UTC m=+62.762220177" watchObservedRunningTime="2026-07-06 14:25:27.221157326 +0000 UTC m=+62.763844864" Jul 06 14:25:29.208508 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:29.208467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" event={"ID":"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e","Type":"ContainerStarted","Data":"e8188e5c12e603336cd03e9475949e5a5e25d34f7520d05ae90504f0884cf43c"} Jul 06 14:25:29.209016 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:29.208528 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" event={"ID":"018655ad-3ef8-43f2-a2b5-cdb7c5718d5e","Type":"ContainerStarted","Data":"3dfb6231d4fe06be67c0be0a82f84535e5057ed2644a0a884eddd4670dd1a35b"} Jul 06 14:25:29.232513 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:29.232461 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-skb8j" podStartSLOduration=17.288535356 podStartE2EDuration="19.232445258s" totalImagesPullingTime="1.943909902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:29.231670161 +0000 UTC m=+64.774357701" watchObservedRunningTime="2026-07-06 14:25:29.232445258 +0000 UTC m=+64.775132846" Jul 06 14:25:30.749638 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.749593 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:25:30.749638 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.749638 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:25:30.752823 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.752798 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:25:30.752823 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.752813 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:25:30.762526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.762499 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcf896e5-cbff-43fa-8d73-b05d9be1a499-original-pull-secret\") pod \"global-pull-secret-syncer-rjl52\" (UID: \"fcf896e5-cbff-43fa-8d73-b05d9be1a499\") " pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:25:30.762613 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.762565 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b04e6a4-fe9b-4b78-954a-aa1927eb521a-metrics-certs\") pod \"network-metrics-daemon-wvgxw\" (UID: \"9b04e6a4-fe9b-4b78-954a-aa1927eb521a\") " pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:25:30.850365 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.850325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:25:30.853477 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.853459 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 14:25:30.864055 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.864037 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 14:25:30.873949 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:30.873929 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s644p\" (UniqueName: \"kubernetes.io/projected/626f1048-dffc-4449-b748-ef625d7de486-kube-api-access-s644p\") pod \"network-check-target-9ls2q\" (UID: \"626f1048-dffc-4449-b748-ef625d7de486\") " pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:25:31.039677 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.039590 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rjl52" Jul 06 14:25:31.050745 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.050724 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fzm8z\"" Jul 06 14:25:31.056667 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.056649 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-689l7\"" Jul 06 14:25:31.058736 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.058718 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:25:31.065067 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.065046 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvgxw" Jul 06 14:25:31.186595 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.186550 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rjl52"] Jul 06 14:25:31.189750 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:31.189723 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf896e5_cbff_43fa_8d73_b05d9be1a499.slice/crio-b47fb47e6f5ec63d2d269e7ff95b9d715a8c5eaefac216af5a0596885e214424 WatchSource:0}: Error finding container b47fb47e6f5ec63d2d269e7ff95b9d715a8c5eaefac216af5a0596885e214424: Status 404 returned error can't find the container with id b47fb47e6f5ec63d2d269e7ff95b9d715a8c5eaefac216af5a0596885e214424 Jul 06 14:25:31.206603 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.206452 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9ls2q"] Jul 06 14:25:31.209129 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:31.209100 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626f1048_dffc_4449_b748_ef625d7de486.slice/crio-64668f53d723d1e4fe6afbc57dfbcd8637e36177c0404f7924396f0766d1dea8 WatchSource:0}: Error finding container 64668f53d723d1e4fe6afbc57dfbcd8637e36177c0404f7924396f0766d1dea8: Status 404 returned error can't find the container with id 64668f53d723d1e4fe6afbc57dfbcd8637e36177c0404f7924396f0766d1dea8 Jul 06 14:25:31.215437 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.215411 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9ls2q" event={"ID":"626f1048-dffc-4449-b748-ef625d7de486","Type":"ContainerStarted","Data":"64668f53d723d1e4fe6afbc57dfbcd8637e36177c0404f7924396f0766d1dea8"} Jul 06 14:25:31.216439 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.216417 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rjl52" event={"ID":"fcf896e5-cbff-43fa-8d73-b05d9be1a499","Type":"ContainerStarted","Data":"b47fb47e6f5ec63d2d269e7ff95b9d715a8c5eaefac216af5a0596885e214424"} Jul 06 14:25:31.229678 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:31.229651 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wvgxw"] Jul 06 14:25:31.232673 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:31.232649 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b04e6a4_fe9b_4b78_954a_aa1927eb521a.slice/crio-68b9ac5af80f553325c1e58bdb9055d1e38e365faff9bf6e31c812aedda1b8b5 WatchSource:0}: Error finding container 68b9ac5af80f553325c1e58bdb9055d1e38e365faff9bf6e31c812aedda1b8b5: Status 404 returned error can't find the container with id 68b9ac5af80f553325c1e58bdb9055d1e38e365faff9bf6e31c812aedda1b8b5 Jul 06 14:25:32.222297 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.222249 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wvgxw" event={"ID":"9b04e6a4-fe9b-4b78-954a-aa1927eb521a","Type":"ContainerStarted","Data":"68b9ac5af80f553325c1e58bdb9055d1e38e365faff9bf6e31c812aedda1b8b5"} Jul 06 14:25:32.666517 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.665939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:32.669464 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.669398 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a04e7fb5-c4dc-48be-9748-428d658f9ca1-metrics-tls\") pod \"dns-default-8psvj\" (UID: \"a04e7fb5-c4dc-48be-9748-428d658f9ca1\") " pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:32.868656 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.868553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:32.868656 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.868616 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:32.869345 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.869294 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:32.872453 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.871659 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0a0a51-50a8-4499-b7dd-c2445bb41b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-whrlv\" (UID: \"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f\") " pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:32.938283 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.938252 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s54bc\"" Jul 06 14:25:32.945952 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:32.945924 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:33.015588 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:33.015375 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mh8w\"" Jul 06 14:25:33.023123 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:33.022856 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-whrlv" Jul 06 14:25:33.123269 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:33.119163 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8psvj"] Jul 06 14:25:33.158358 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:33.157565 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04e7fb5_c4dc_48be_9748_428d658f9ca1.slice/crio-6e94afd5363ca3a7c9385ba54269d77129dca0ba6bb2016edc4fe630c19f2635 WatchSource:0}: Error finding container 6e94afd5363ca3a7c9385ba54269d77129dca0ba6bb2016edc4fe630c19f2635: Status 404 returned error can't find the container with id 6e94afd5363ca3a7c9385ba54269d77129dca0ba6bb2016edc4fe630c19f2635 Jul 06 14:25:33.229461 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:33.229421 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8psvj" event={"ID":"a04e7fb5-c4dc-48be-9748-428d658f9ca1","Type":"ContainerStarted","Data":"6e94afd5363ca3a7c9385ba54269d77129dca0ba6bb2016edc4fe630c19f2635"} Jul 06 14:25:33.232609 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:33.232340 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wvgxw" event={"ID":"9b04e6a4-fe9b-4b78-954a-aa1927eb521a","Type":"ContainerStarted","Data":"abb394e3139bb66ea5999d51c1423c2e40138d80c98e4a92e1e8911ccef23f95"} Jul 06 14:25:33.232609 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:33.232407 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wvgxw" event={"ID":"9b04e6a4-fe9b-4b78-954a-aa1927eb521a","Type":"ContainerStarted","Data":"98f1866f0e1a4d7e5573aed6f06994b529dd07f61e4b9dde08239e7bddcc2964"} Jul 06 14:25:33.252721 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:33.252666 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wvgxw" podStartSLOduration=67.090637785 podStartE2EDuration="1m8.252645868s" totalImagesPullingTime="1.162008083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:33.251971401 +0000 UTC m=+68.794658939" watchObservedRunningTime="2026-07-06 14:25:33.252645868 +0000 UTC m=+68.795333406" Jul 06 14:25:33.256879 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:33.256849 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-whrlv"] Jul 06 14:25:33.260360 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:33.260333 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0a0a51_50a8_4499_b7dd_c2445bb41b8f.slice/crio-ef4bcd8ef37c7288ca06013e7b2751dad7f8915c82e684d271ed0861143387f6 WatchSource:0}: Error finding container ef4bcd8ef37c7288ca06013e7b2751dad7f8915c82e684d271ed0861143387f6: Status 404 returned error can't find the container with id ef4bcd8ef37c7288ca06013e7b2751dad7f8915c82e684d271ed0861143387f6 Jul 06 14:25:34.238868 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:34.238831 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-whrlv" event={"ID":"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f","Type":"ContainerStarted","Data":"ef4bcd8ef37c7288ca06013e7b2751dad7f8915c82e684d271ed0861143387f6"} Jul 06 14:25:37.259102 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.259061 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rjl52" event={"ID":"fcf896e5-cbff-43fa-8d73-b05d9be1a499","Type":"ContainerStarted","Data":"ad29382dcbddbd25963739f8eb439336fade0aac49ee2d66c8e4a1b90e059fc8"} Jul 06 14:25:37.261343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.261311 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8psvj" event={"ID":"a04e7fb5-c4dc-48be-9748-428d658f9ca1","Type":"ContainerStarted","Data":"e63007123950fb6a742f8658bcbc2717dfc6861ef5ceb46607e286d0628227d1"} Jul 06 14:25:37.261471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.261366 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8psvj" event={"ID":"a04e7fb5-c4dc-48be-9748-428d658f9ca1","Type":"ContainerStarted","Data":"fc8a32353a701fb1afce9380448e52d981ba0165af0d703fd7da7155b5929f53"} Jul 06 14:25:37.261471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.261396 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:37.262998 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.262971 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9ls2q" event={"ID":"626f1048-dffc-4449-b748-ef625d7de486","Type":"ContainerStarted","Data":"0d2d4d6bd1a0a82d99e66600263c12c25120fbad1ff856e2b362846ad36b7d62"} Jul 06 14:25:37.263432 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.263416 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:25:37.296266 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.296213 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rjl52" podStartSLOduration=66.583903774 podStartE2EDuration="1m12.296196079s" totalImagesPullingTime="5.712292305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:37.2767137 +0000 UTC m=+72.819401243" watchObservedRunningTime="2026-07-06 14:25:37.296196079 +0000 UTC m=+72.838883611" Jul 06 14:25:37.297667 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.297622 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9ls2q" podStartSLOduration=66.605430098 podStartE2EDuration="1m12.297607324s" totalImagesPullingTime="5.692177226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:37.296327074 +0000 UTC m=+72.839014619" watchObservedRunningTime="2026-07-06 14:25:37.297607324 +0000 UTC m=+72.840294866" Jul 06 14:25:37.316180 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.316131 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8psvj" podStartSLOduration=33.51793455 podStartE2EDuration="37.316113079s" totalImagesPullingTime="3.798178529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:37.314990889 +0000 UTC m=+72.857678427" watchObservedRunningTime="2026-07-06 14:25:37.316113079 +0000 UTC m=+72.858800618" Jul 06 14:25:37.408998 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.408961 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp"] Jul 06 14:25:37.412420 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.412399 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2"] Jul 06 14:25:37.412547 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.412538 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.415459 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.415400 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.416170 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.416150 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 14:25:37.416276 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.416174 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 14:25:37.416276 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.416216 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 14:25:37.416529 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.416510 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 14:25:37.422929 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.422913 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 14:25:37.424596 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.424577 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 14:25:37.424871 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.424851 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 14:25:37.424960 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.424892 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 14:25:37.432178 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.432151 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2"] Jul 06 14:25:37.436207 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.436186 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp"] Jul 06 14:25:37.508547 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508516 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.508547 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508552 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwmkv\" (UniqueName: \"kubernetes.io/projected/c31ee539-031a-4aff-87f0-ae637deb3905-kube-api-access-lwmkv\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.508778 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508580 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jfn7\" (UniqueName: \"kubernetes.io/projected/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-kube-api-access-2jfn7\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.508778 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508662 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31ee539-031a-4aff-87f0-ae637deb3905-tmp\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.508778 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.508778 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508743 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-hub\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.509054 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508841 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.509054 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508879 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c31ee539-031a-4aff-87f0-ae637deb3905-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.509054 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.508901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-ca\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.511346 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.511321 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-585f7947c-llrbp"] Jul 06 14:25:37.548882 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.548839 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d75689b57-26vdj"] Jul 06 14:25:37.553551 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.553523 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.565509 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.565484 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d75689b57-26vdj"] Jul 06 14:25:37.609929 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.609897 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-ca\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.610079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.609942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b873ff1-9a75-4ee5-9986-c3adca081c49-registry-certificates\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.610153 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610100 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.610153 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-registry-tls\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.610253 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610156 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-bound-sa-token\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.610253 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610185 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b873ff1-9a75-4ee5-9986-c3adca081c49-trusted-ca\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.610353 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610314 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwmkv\" (UniqueName: \"kubernetes.io/projected/c31ee539-031a-4aff-87f0-ae637deb3905-kube-api-access-lwmkv\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.610402 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610356 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b873ff1-9a75-4ee5-9986-c3adca081c49-installation-pull-secrets\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.610402 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610394 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jfn7\" (UniqueName: \"kubernetes.io/projected/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-kube-api-access-2jfn7\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.610506 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b873ff1-9a75-4ee5-9986-c3adca081c49-image-registry-private-configuration\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.610506 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610445 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31ee539-031a-4aff-87f0-ae637deb3905-tmp\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.610506 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.610506 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610482 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpwmk\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-kube-api-access-mpwmk\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.610686 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-hub\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.610686 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610550 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.610686 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610577 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b873ff1-9a75-4ee5-9986-c3adca081c49-ca-trust-extracted\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.610686 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.610608 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c31ee539-031a-4aff-87f0-ae637deb3905-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.611272 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.611098 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.611384 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.611297 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31ee539-031a-4aff-87f0-ae637deb3905-tmp\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.613311 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.613283 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-ca\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.613754 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.613736 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.613922 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.613894 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-hub\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.614039 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.613927 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c31ee539-031a-4aff-87f0-ae637deb3905-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.614450 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.614426 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.628676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.628649 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwmkv\" (UniqueName: \"kubernetes.io/projected/c31ee539-031a-4aff-87f0-ae637deb3905-kube-api-access-lwmkv\") pod \"klusterlet-addon-workmgr-6b757f599f-rlnr2\" (UID: \"c31ee539-031a-4aff-87f0-ae637deb3905\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.629390 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.629364 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jfn7\" (UniqueName: \"kubernetes.io/projected/1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681-kube-api-access-2jfn7\") pod \"cluster-proxy-proxy-agent-c4dc88f5-9j2fp\" (UID: \"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.711735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpwmk\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-kube-api-access-mpwmk\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.711735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711381 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b873ff1-9a75-4ee5-9986-c3adca081c49-ca-trust-extracted\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.711735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b873ff1-9a75-4ee5-9986-c3adca081c49-registry-certificates\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.711735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711457 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-registry-tls\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.711735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711481 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-bound-sa-token\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.711735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711511 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b873ff1-9a75-4ee5-9986-c3adca081c49-trusted-ca\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.711735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711555 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b873ff1-9a75-4ee5-9986-c3adca081c49-installation-pull-secrets\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.711735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711586 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b873ff1-9a75-4ee5-9986-c3adca081c49-image-registry-private-configuration\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.712309 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.711814 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b873ff1-9a75-4ee5-9986-c3adca081c49-ca-trust-extracted\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.712396 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.712369 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b873ff1-9a75-4ee5-9986-c3adca081c49-registry-certificates\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.712564 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.712545 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b873ff1-9a75-4ee5-9986-c3adca081c49-trusted-ca\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.715184 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.715163 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b873ff1-9a75-4ee5-9986-c3adca081c49-installation-pull-secrets\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.715277 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.715185 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b873ff1-9a75-4ee5-9986-c3adca081c49-image-registry-private-configuration\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.715412 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.715394 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-registry-tls\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.721238 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.721199 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-bound-sa-token\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.721760 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.721742 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpwmk\" (UniqueName: \"kubernetes.io/projected/4b873ff1-9a75-4ee5-9986-c3adca081c49-kube-api-access-mpwmk\") pod \"image-registry-d75689b57-26vdj\" (UID: \"4b873ff1-9a75-4ee5-9986-c3adca081c49\") " pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:37.729696 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.729674 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" Jul 06 14:25:37.736900 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.736825 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:37.864279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:37.864250 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:39.863840 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:39.863621 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d75689b57-26vdj"] Jul 06 14:25:39.867355 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:39.867322 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b873ff1_9a75_4ee5_9986_c3adca081c49.slice/crio-490bcfc9de2040b2a041986e8a61a9592d2b97be706c6e6d8d40bb6fea70b30d WatchSource:0}: Error finding container 490bcfc9de2040b2a041986e8a61a9592d2b97be706c6e6d8d40bb6fea70b30d: Status 404 returned error can't find the container with id 490bcfc9de2040b2a041986e8a61a9592d2b97be706c6e6d8d40bb6fea70b30d Jul 06 14:25:40.076529 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.076450 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp"] Jul 06 14:25:40.079115 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.079090 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2"] Jul 06 14:25:40.080438 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:40.080411 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b25aa4a_a2cf_4b6f_99c3_f7bf2d261681.slice/crio-534aa02de7b3444b0c714f2114a0c25e26f69084a0d01999f408071c91c4929b WatchSource:0}: Error finding container 534aa02de7b3444b0c714f2114a0c25e26f69084a0d01999f408071c91c4929b: Status 404 returned error can't find the container with id 534aa02de7b3444b0c714f2114a0c25e26f69084a0d01999f408071c91c4929b Jul 06 14:25:40.081974 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:40.081946 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc31ee539_031a_4aff_87f0_ae637deb3905.slice/crio-a00b0067d0cd58103b14aee880a4688cea69bf98179b976c2cf689b0918b06ea WatchSource:0}: Error finding container a00b0067d0cd58103b14aee880a4688cea69bf98179b976c2cf689b0918b06ea: Status 404 returned error can't find the container with id a00b0067d0cd58103b14aee880a4688cea69bf98179b976c2cf689b0918b06ea Jul 06 14:25:40.276473 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.276429 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" event={"ID":"c31ee539-031a-4aff-87f0-ae637deb3905","Type":"ContainerStarted","Data":"a00b0067d0cd58103b14aee880a4688cea69bf98179b976c2cf689b0918b06ea"} Jul 06 14:25:40.277600 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.277575 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" event={"ID":"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681","Type":"ContainerStarted","Data":"534aa02de7b3444b0c714f2114a0c25e26f69084a0d01999f408071c91c4929b"} Jul 06 14:25:40.278999 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.278974 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d75689b57-26vdj" event={"ID":"4b873ff1-9a75-4ee5-9986-c3adca081c49","Type":"ContainerStarted","Data":"35f70f607198f4e24348f3c4dc72d2762e02b3b6af32f156540f2b61ee15e78d"} Jul 06 14:25:40.279115 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.279018 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d75689b57-26vdj" event={"ID":"4b873ff1-9a75-4ee5-9986-c3adca081c49","Type":"ContainerStarted","Data":"490bcfc9de2040b2a041986e8a61a9592d2b97be706c6e6d8d40bb6fea70b30d"} Jul 06 14:25:40.279115 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.279038 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:25:40.280270 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.280252 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-whrlv" event={"ID":"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f","Type":"ContainerStarted","Data":"f1d32b23c5cf6137fff2ec5133fba01095792175808616e4cd6eb95953323b5f"} Jul 06 14:25:40.302547 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:40.302504 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d75689b57-26vdj" podStartSLOduration=3.30249094 podStartE2EDuration="3.30249094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:40.300935204 +0000 UTC m=+75.843622745" watchObservedRunningTime="2026-07-06 14:25:40.30249094 +0000 UTC m=+75.845178456" Jul 06 14:25:41.287503 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:41.287416 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-whrlv" event={"ID":"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f","Type":"ContainerStarted","Data":"b31305fce879c69c806ea0ecf092cc6e6cad0a7a82e04bb4eec8426d72eac3c1"} Jul 06 14:25:42.152195 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:42.152152 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:42.152925 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:42.152898 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e221b5b-b395-4184-9dfd-709037b5d5fd-service-ca-bundle\") pod \"router-default-5477b4dcdf-nnrp4\" (UID: \"5e221b5b-b395-4184-9dfd-709037b5d5fd\") " pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:42.355833 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:42.355640 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-m9tb9\"" Jul 06 14:25:42.363156 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:42.363126 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:45.422178 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:45.422152 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5477b4dcdf-nnrp4"] Jul 06 14:25:45.425242 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:45.425218 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e221b5b_b395_4184_9dfd_709037b5d5fd.slice/crio-ad48d88947953d001c8ea91de1ed073fa0324798159f52fc628b70a0d24938c5 WatchSource:0}: Error finding container ad48d88947953d001c8ea91de1ed073fa0324798159f52fc628b70a0d24938c5: Status 404 returned error can't find the container with id ad48d88947953d001c8ea91de1ed073fa0324798159f52fc628b70a0d24938c5 Jul 06 14:25:46.308551 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.308511 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-whrlv" event={"ID":"8f0a0a51-50a8-4499-b7dd-c2445bb41b8f","Type":"ContainerStarted","Data":"1cd55ed5156589cf7d1fe5ff6bfef22ea604046151c77eaf196d038d77d3e1b7"} Jul 06 14:25:46.310380 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.310349 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" event={"ID":"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681","Type":"ContainerStarted","Data":"f1c5d83a383dd29f77c265ab92d3f01adf8f5eae18c477e0957cdb5c9fa02e8d"} Jul 06 14:25:46.310497 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.310389 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" event={"ID":"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681","Type":"ContainerStarted","Data":"99ed2fb0783376ded852162f2d1b53b44c62f97ac17ae36332bc62df634ce1ef"} Jul 06 14:25:46.310497 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.310402 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" event={"ID":"1b25aa4a-a2cf-4b6f-99c3-f7bf2d261681","Type":"ContainerStarted","Data":"ff21b713d97e3154ba96f4945589c35a6665cf144a777e157239154d5a204874"} Jul 06 14:25:46.311602 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.311583 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" event={"ID":"5e221b5b-b395-4184-9dfd-709037b5d5fd","Type":"ContainerStarted","Data":"484afa92045f646f47ecab6e96bdd7e8073ced1c66dd276a965603917ac78c62"} Jul 06 14:25:46.311668 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.311618 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" event={"ID":"5e221b5b-b395-4184-9dfd-709037b5d5fd","Type":"ContainerStarted","Data":"ad48d88947953d001c8ea91de1ed073fa0324798159f52fc628b70a0d24938c5"} Jul 06 14:25:46.327637 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.327586 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-whrlv" podStartSLOduration=17.444796416 podStartE2EDuration="29.327568033s" totalImagesPullingTime="11.882771617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:25:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:46.327521007 +0000 UTC m=+81.870208544" watchObservedRunningTime="2026-07-06 14:25:46.327568033 +0000 UTC m=+81.870255574" Jul 06 14:25:46.354608 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.354560 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" podStartSLOduration=36.354545731 podStartE2EDuration="36.354545731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:46.352940037 +0000 UTC m=+81.895627575" watchObservedRunningTime="2026-07-06 14:25:46.354545731 +0000 UTC m=+81.897233268" Jul 06 14:25:46.364091 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.364061 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:46.366604 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.366582 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:46.374592 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:46.374519 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c4dc88f5-9j2fp" podStartSLOduration=4.16297864 podStartE2EDuration="9.374503764s" totalImagesPullingTime="5.211525124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:46.373675837 +0000 UTC m=+81.916363377" watchObservedRunningTime="2026-07-06 14:25:46.374503764 +0000 UTC m=+81.917191352" Jul 06 14:25:47.270208 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:47.270181 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8psvj" Jul 06 14:25:47.317030 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:47.317000 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:47.318454 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:47.318431 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5477b4dcdf-nnrp4" Jul 06 14:25:47.517539 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:47.517504 2565 patch_prober.go:28] interesting pod/image-registry-585f7947c-llrbp container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 14:25:47.517707 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:47.517560 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-585f7947c-llrbp" podUID="64c63ad8-e362-4adc-bfc7-c3b688665c8f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:25:48.322043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:48.322005 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" event={"ID":"c31ee539-031a-4aff-87f0-ae637deb3905","Type":"ContainerStarted","Data":"f99532e080a1238c692722da51dbb0b53da518426cd35534427f02c864d91958"} Jul 06 14:25:48.339461 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:48.339407 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" podStartSLOduration=3.752302829 podStartE2EDuration="11.339390508s" totalImagesPullingTime="7.587087679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:48.338961564 +0000 UTC m=+83.881649104" watchObservedRunningTime="2026-07-06 14:25:48.339390508 +0000 UTC m=+83.882078045" Jul 06 14:25:49.325546 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:49.325507 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:49.327408 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:49.327386 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b757f599f-rlnr2" Jul 06 14:25:56.473971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.473939 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-64lkd"] Jul 06 14:25:56.478880 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.478858 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.481999 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.481973 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 14:25:56.482131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.482026 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 14:25:56.482201 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.482182 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 14:25:56.482248 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.482202 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 14:25:56.483389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.483373 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 14:25:56.483521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.483440 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5g2jr\"" Jul 06 14:25:56.483521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.483443 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 14:25:56.570504 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570471 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxb75\" (UniqueName: \"kubernetes.io/projected/d667376e-1eea-44d0-b0ef-ce35edc8e783-kube-api-access-pxb75\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.570504 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570506 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-accelerators-collector-config\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.570757 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570539 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-textfile\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.570757 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570559 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d667376e-1eea-44d0-b0ef-ce35edc8e783-metrics-client-ca\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.570757 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570622 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-tls\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.570757 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570645 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-sys\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.570757 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.570757 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570726 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-wtmp\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.570757 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.570748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-root\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671227 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671198 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-textfile\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671227 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671229 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d667376e-1eea-44d0-b0ef-ce35edc8e783-metrics-client-ca\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-tls\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671275 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-sys\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671303 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671328 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-wtmp\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671346 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-root\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671374 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxb75\" (UniqueName: \"kubernetes.io/projected/d667376e-1eea-44d0-b0ef-ce35edc8e783-kube-api-access-pxb75\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.671471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671391 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-accelerators-collector-config\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.672008 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671865 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-accelerators-collector-config\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.672008 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.671927 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-sys\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.672206 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.672176 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-root\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.672438 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.672389 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-textfile\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.672746 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.672725 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-wtmp\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.672865 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.672846 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d667376e-1eea-44d0-b0ef-ce35edc8e783-metrics-client-ca\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.672955 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:56.672940 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 14:25:56.673023 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:25:56.673012 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-tls podName:d667376e-1eea-44d0-b0ef-ce35edc8e783 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:57.172992442 +0000 UTC m=+92.715679965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-tls") pod "node-exporter-64lkd" (UID: "d667376e-1eea-44d0-b0ef-ce35edc8e783") : secret "node-exporter-tls" not found Jul 06 14:25:56.674504 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.674451 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:56.684169 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:56.684139 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxb75\" (UniqueName: \"kubernetes.io/projected/d667376e-1eea-44d0-b0ef-ce35edc8e783-kube-api-access-pxb75\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:57.176962 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:57.176916 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-tls\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:57.179480 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:57.179455 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d667376e-1eea-44d0-b0ef-ce35edc8e783-node-exporter-tls\") pod \"node-exporter-64lkd\" (UID: \"d667376e-1eea-44d0-b0ef-ce35edc8e783\") " pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:57.389070 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:57.389031 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-64lkd" Jul 06 14:25:57.401637 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:25:57.401591 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd667376e_1eea_44d0_b0ef_ce35edc8e783.slice/crio-ba3599fd51d66da6a6e8a4e03cea77bbcd7b307c64b0c756dc1226b8271ffa20 WatchSource:0}: Error finding container ba3599fd51d66da6a6e8a4e03cea77bbcd7b307c64b0c756dc1226b8271ffa20: Status 404 returned error can't find the container with id ba3599fd51d66da6a6e8a4e03cea77bbcd7b307c64b0c756dc1226b8271ffa20 Jul 06 14:25:57.515713 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:57.515682 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:25:58.358729 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:58.358683 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-64lkd" event={"ID":"d667376e-1eea-44d0-b0ef-ce35edc8e783","Type":"ContainerStarted","Data":"ba3599fd51d66da6a6e8a4e03cea77bbcd7b307c64b0c756dc1226b8271ffa20"} Jul 06 14:25:59.368204 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:59.368164 2565 generic.go:356] "Generic (PLEG): container finished" podID="d667376e-1eea-44d0-b0ef-ce35edc8e783" containerID="263f194c4b77b28f5e439db72a4bcc2513847fbf423c09ce674745280eb9b8e7" exitCode=0 Jul 06 14:25:59.368760 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:25:59.368223 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-64lkd" event={"ID":"d667376e-1eea-44d0-b0ef-ce35edc8e783","Type":"ContainerDied","Data":"263f194c4b77b28f5e439db72a4bcc2513847fbf423c09ce674745280eb9b8e7"} Jul 06 14:26:00.374176 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:00.374137 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-64lkd" event={"ID":"d667376e-1eea-44d0-b0ef-ce35edc8e783","Type":"ContainerStarted","Data":"833bf8ce2c30ab01c4946125dce2f401733674598f82cfe7d970aad98c7079eb"} Jul 06 14:26:00.374541 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:00.374210 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-64lkd" event={"ID":"d667376e-1eea-44d0-b0ef-ce35edc8e783","Type":"ContainerStarted","Data":"29b3502c1d8165f1f651479d8ba89254f72c687ef56e3efa570d42259cf2054e"} Jul 06 14:26:00.397680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:00.397628 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-64lkd" podStartSLOduration=3.243755637 podStartE2EDuration="4.397613634s" totalImagesPullingTime="1.080444747s" totalInitContainerRuntime="73.41325ms" isStatefulPod=true podCreationTimestamp="2026-07-06 14:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:26:00.395696303 +0000 UTC m=+95.938383842" watchObservedRunningTime="2026-07-06 14:26:00.397613634 +0000 UTC m=+95.940301172" Jul 06 14:26:01.292256 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:01.292222 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d75689b57-26vdj" Jul 06 14:26:02.532372 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.532324 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-585f7947c-llrbp" podUID="64c63ad8-e362-4adc-bfc7-c3b688665c8f" containerName="registry" containerID="cri-o://552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e" gracePeriod=30 Jul 06 14:26:02.780971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.780948 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:26:02.924914 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.924821 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-installation-pull-secrets\") pod \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " Jul 06 14:26:02.924914 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.924865 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c63ad8-e362-4adc-bfc7-c3b688665c8f-ca-trust-extracted\") pod \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " Jul 06 14:26:02.924914 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.924891 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbvld\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-kube-api-access-kbvld\") pod \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " Jul 06 14:26:02.924914 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.924915 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-certificates\") pod \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " Jul 06 14:26:02.925219 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.924984 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") pod \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " Jul 06 14:26:02.925219 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.925024 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-image-registry-private-configuration\") pod \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " Jul 06 14:26:02.925219 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.925069 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-trusted-ca\") pod \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " Jul 06 14:26:02.925219 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.925100 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-bound-sa-token\") pod \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\" (UID: \"64c63ad8-e362-4adc-bfc7-c3b688665c8f\") " Jul 06 14:26:02.925690 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.925459 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "64c63ad8-e362-4adc-bfc7-c3b688665c8f" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:26:02.925690 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.925681 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "64c63ad8-e362-4adc-bfc7-c3b688665c8f" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:26:02.928104 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.928047 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "64c63ad8-e362-4adc-bfc7-c3b688665c8f" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:26:02.928220 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.928124 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-kube-api-access-kbvld" (OuterVolumeSpecName: "kube-api-access-kbvld") pod "64c63ad8-e362-4adc-bfc7-c3b688665c8f" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f"). InnerVolumeSpecName "kube-api-access-kbvld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:26:02.928220 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.928169 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "64c63ad8-e362-4adc-bfc7-c3b688665c8f" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:26:02.928326 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.928249 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "64c63ad8-e362-4adc-bfc7-c3b688665c8f" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:26:02.928539 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.928505 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "64c63ad8-e362-4adc-bfc7-c3b688665c8f" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:26:02.936406 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:02.936372 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c63ad8-e362-4adc-bfc7-c3b688665c8f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "64c63ad8-e362-4adc-bfc7-c3b688665c8f" (UID: "64c63ad8-e362-4adc-bfc7-c3b688665c8f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:26:03.026717 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.026681 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-trusted-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:26:03.026717 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.026715 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-bound-sa-token\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:26:03.026958 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.026730 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-installation-pull-secrets\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:26:03.026958 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.026744 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c63ad8-e362-4adc-bfc7-c3b688665c8f-ca-trust-extracted\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:26:03.026958 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.026757 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbvld\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-kube-api-access-kbvld\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:26:03.026958 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.026769 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-certificates\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:26:03.026958 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.026800 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c63ad8-e362-4adc-bfc7-c3b688665c8f-registry-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:26:03.026958 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.026814 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c63ad8-e362-4adc-bfc7-c3b688665c8f-image-registry-private-configuration\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:26:03.390553 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.390512 2565 generic.go:356] "Generic (PLEG): container finished" podID="64c63ad8-e362-4adc-bfc7-c3b688665c8f" containerID="552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e" exitCode=0 Jul 06 14:26:03.390734 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.390565 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-585f7947c-llrbp" Jul 06 14:26:03.390734 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.390586 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-585f7947c-llrbp" event={"ID":"64c63ad8-e362-4adc-bfc7-c3b688665c8f","Type":"ContainerDied","Data":"552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e"} Jul 06 14:26:03.390734 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.390653 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-585f7947c-llrbp" event={"ID":"64c63ad8-e362-4adc-bfc7-c3b688665c8f","Type":"ContainerDied","Data":"7019326a695ee21514abec7d8130966c2b4fdc950dd530f2e1729e36a09f5f44"} Jul 06 14:26:03.390734 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.390677 2565 scope.go:117] "RemoveContainer" containerID="552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e" Jul 06 14:26:03.399622 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.399606 2565 scope.go:117] "RemoveContainer" containerID="552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e" Jul 06 14:26:03.399893 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:26:03.399871 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e\": container with ID starting with 552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e not found: ID does not exist" containerID="552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e" Jul 06 14:26:03.399953 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.399900 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e"} err="failed to get container status \"552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e\": rpc error: code = NotFound desc = could not find container \"552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e\": container with ID starting with 552ea9f646f2b3e8a8077ae513a6960523104dafe408a693648a8bee3082685e not found: ID does not exist" Jul 06 14:26:03.414936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.414911 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-585f7947c-llrbp"] Jul 06 14:26:03.419358 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:03.419331 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-585f7947c-llrbp"] Jul 06 14:26:04.931194 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:04.931162 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c63ad8-e362-4adc-bfc7-c3b688665c8f" path="/var/lib/kubelet/pods/64c63ad8-e362-4adc-bfc7-c3b688665c8f/volumes" Jul 06 14:26:09.273385 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:09.273355 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9ls2q" Jul 06 14:26:35.497231 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:35.497194 2565 generic.go:356] "Generic (PLEG): container finished" podID="d7f9c177-4e04-4842-9c10-828e0cc83929" containerID="3346074c9e3e91bde0d81cd34c9ce51e07190b0697b212f0b861baeb319425a2" exitCode=0 Jul 06 14:26:35.497641 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:35.497266 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" event={"ID":"d7f9c177-4e04-4842-9c10-828e0cc83929","Type":"ContainerDied","Data":"3346074c9e3e91bde0d81cd34c9ce51e07190b0697b212f0b861baeb319425a2"} Jul 06 14:26:35.497641 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:35.497569 2565 scope.go:117] "RemoveContainer" containerID="3346074c9e3e91bde0d81cd34c9ce51e07190b0697b212f0b861baeb319425a2" Jul 06 14:26:36.503425 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:26:36.503391 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-ss27p" event={"ID":"d7f9c177-4e04-4842-9c10-828e0cc83929","Type":"ContainerStarted","Data":"9a5974b5245efd6e223edc57cee598929e653c36f46b65cc2f2b3866c6f0601c"} Jul 06 14:29:16.954763 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.954729 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w"] Jul 06 14:29:16.955220 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.954989 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c63ad8-e362-4adc-bfc7-c3b688665c8f" containerName="registry" Jul 06 14:29:16.955220 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.955001 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c63ad8-e362-4adc-bfc7-c3b688665c8f" containerName="registry" Jul 06 14:29:16.955220 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.955058 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c63ad8-e362-4adc-bfc7-c3b688665c8f" containerName="registry" Jul 06 14:29:16.957833 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.957818 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:16.964499 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.964090 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 14:29:16.964499 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.964123 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 14:29:16.964499 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.964134 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 14:29:16.964499 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.964127 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 14:29:16.964499 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.964299 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qlzj2\"" Jul 06 14:29:16.967975 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.967948 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a1d148e-781a-4253-a4f7-b3b6805208f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:16.968083 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.967984 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q576l\" (UniqueName: \"kubernetes.io/projected/3a1d148e-781a-4253-a4f7-b3b6805208f0-kube-api-access-q576l\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:16.968083 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.968054 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3a1d148e-781a-4253-a4f7-b3b6805208f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:16.968083 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.968078 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3a1d148e-781a-4253-a4f7-b3b6805208f0-manifests\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:16.968202 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.968096 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a1d148e-781a-4253-a4f7-b3b6805208f0-tmp\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:16.978839 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:16.978817 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w"] Jul 06 14:29:17.068801 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.068761 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a1d148e-781a-4253-a4f7-b3b6805208f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.068994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.068823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q576l\" (UniqueName: \"kubernetes.io/projected/3a1d148e-781a-4253-a4f7-b3b6805208f0-kube-api-access-q576l\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.068994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.068880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3a1d148e-781a-4253-a4f7-b3b6805208f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.068994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.068898 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3a1d148e-781a-4253-a4f7-b3b6805208f0-manifests\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.068994 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.068914 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a1d148e-781a-4253-a4f7-b3b6805208f0-tmp\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.069360 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.069337 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3a1d148e-781a-4253-a4f7-b3b6805208f0-manifests\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.071273 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.071255 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a1d148e-781a-4253-a4f7-b3b6805208f0-tmp\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.071607 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.071587 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3a1d148e-781a-4253-a4f7-b3b6805208f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.071646 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.071607 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a1d148e-781a-4253-a4f7-b3b6805208f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.083685 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.083664 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q576l\" (UniqueName: \"kubernetes.io/projected/3a1d148e-781a-4253-a4f7-b3b6805208f0-kube-api-access-q576l\") pod \"opendatahub-operator-controller-manager-9447c645c-nxt5w\" (UID: \"3a1d148e-781a-4253-a4f7-b3b6805208f0\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.268386 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.268290 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:17.411755 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:17.411728 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w"] Jul 06 14:29:17.414221 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:29:17.414191 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a1d148e_781a_4253_a4f7_b3b6805208f0.slice/crio-f104374208af1057ab090d9a9a27706862b59352edc2d2b8da807f16947e72e3 WatchSource:0}: Error finding container f104374208af1057ab090d9a9a27706862b59352edc2d2b8da807f16947e72e3: Status 404 returned error can't find the container with id f104374208af1057ab090d9a9a27706862b59352edc2d2b8da807f16947e72e3 Jul 06 14:29:18.059719 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:18.059649 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" event={"ID":"3a1d148e-781a-4253-a4f7-b3b6805208f0","Type":"ContainerStarted","Data":"f104374208af1057ab090d9a9a27706862b59352edc2d2b8da807f16947e72e3"} Jul 06 14:29:21.077633 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:21.077589 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" event={"ID":"3a1d148e-781a-4253-a4f7-b3b6805208f0","Type":"ContainerStarted","Data":"5c303ed7fe3626006a6b73c53c040715a31b3f95f9a8246648808ff5518a4e75"} Jul 06 14:29:22.082305 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:22.082269 2565 generic.go:356] "Generic (PLEG): container finished" podID="3a1d148e-781a-4253-a4f7-b3b6805208f0" containerID="5c303ed7fe3626006a6b73c53c040715a31b3f95f9a8246648808ff5518a4e75" exitCode=0 Jul 06 14:29:22.082812 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:22.082319 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" event={"ID":"3a1d148e-781a-4253-a4f7-b3b6805208f0","Type":"ContainerDied","Data":"5c303ed7fe3626006a6b73c53c040715a31b3f95f9a8246648808ff5518a4e75"} Jul 06 14:29:23.088109 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:23.088074 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" event={"ID":"3a1d148e-781a-4253-a4f7-b3b6805208f0","Type":"ContainerStarted","Data":"10d14b47e6d190265d14f082e713b51c57d5d40dff7b7884512b977e743f69df"} Jul 06 14:29:23.088489 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:23.088199 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:23.123839 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:23.123766 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" podStartSLOduration=2.820899148 podStartE2EDuration="7.123749879s" totalImagesPullingTime="3.245915548s" totalInitContainerRuntime="1.056935183s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:23.121514155 +0000 UTC m=+298.664201692" watchObservedRunningTime="2026-07-06 14:29:23.123749879 +0000 UTC m=+298.666437417" Jul 06 14:29:24.829956 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:24.829929 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 14:29:34.094209 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:34.094179 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-nxt5w" Jul 06 14:29:42.975065 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:42.975029 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm"] Jul 06 14:29:42.977957 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:42.977940 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:42.980760 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:42.980739 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 14:29:42.980760 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:42.980750 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 14:29:42.980942 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:42.980845 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 14:29:42.981032 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:42.981018 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 14:29:42.981093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:42.981073 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lnrf7\"" Jul 06 14:29:42.992686 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:42.992664 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm"] Jul 06 14:29:43.039836 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.039767 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-token\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.039836 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.039838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.040093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.039888 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.040093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.039927 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-cacerts\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.040093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.039957 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6eb69342-99d1-44ca-ae13-077e565f71c9-local-certs\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.040093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.040028 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.040093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.040050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5wpb\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-kube-api-access-g5wpb\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.141259 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.141217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.141259 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.141260 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-cacerts\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.141526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.141284 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6eb69342-99d1-44ca-ae13-077e565f71c9-local-certs\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.141526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.141325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.141526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.141356 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5wpb\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-kube-api-access-g5wpb\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.141526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.141395 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-token\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.141526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.141419 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.141936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.141915 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.143822 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.143777 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.143941 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.143840 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.144125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.144106 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6eb69342-99d1-44ca-ae13-077e565f71c9-local-certs\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.144225 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.144211 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-cacerts\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.150585 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.150560 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-token\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.150833 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.150808 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5wpb\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-kube-api-access-g5wpb\") pod \"istiod-openshift-gateway-7574459656-4dqgm\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.286773 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.286680 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:43.426413 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.426385 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm"] Jul 06 14:29:43.428725 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:29:43.428695 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb69342_99d1_44ca_ae13_077e565f71c9.slice/crio-95bd9b1731d2ba81c92c5d8e84e95724a78b3e292f37fd8cb12aecafcac5222a WatchSource:0}: Error finding container 95bd9b1731d2ba81c92c5d8e84e95724a78b3e292f37fd8cb12aecafcac5222a: Status 404 returned error can't find the container with id 95bd9b1731d2ba81c92c5d8e84e95724a78b3e292f37fd8cb12aecafcac5222a Jul 06 14:29:43.431126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:43.431107 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:29:44.165167 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.165108 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" event={"ID":"6eb69342-99d1-44ca-ae13-077e565f71c9","Type":"ContainerStarted","Data":"95bd9b1731d2ba81c92c5d8e84e95724a78b3e292f37fd8cb12aecafcac5222a"} Jul 06 14:29:44.457637 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.457603 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2"] Jul 06 14:29:44.461005 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.460982 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.532571 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.532539 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2"] Jul 06 14:29:44.553484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.553458 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.553655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.553486 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.553655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.553519 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.553655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.553541 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.553655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.553573 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.553655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.553604 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b23543c-75ef-4b27-8e6f-40a470c76d90-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.553867 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.553660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjjbf\" (UniqueName: \"kubernetes.io/projected/8b23543c-75ef-4b27-8e6f-40a470c76d90-kube-api-access-wjjbf\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.654985 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.654951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.654985 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.654984 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.655174 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.655018 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.655174 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.655050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b23543c-75ef-4b27-8e6f-40a470c76d90-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.655174 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.655108 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjjbf\" (UniqueName: \"kubernetes.io/projected/8b23543c-75ef-4b27-8e6f-40a470c76d90-kube-api-access-wjjbf\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.655174 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.655132 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.655174 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.655146 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.655804 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.655766 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.657482 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.657460 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b23543c-75ef-4b27-8e6f-40a470c76d90-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.657637 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.657615 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.658015 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.657999 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.658062 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.658006 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b23543c-75ef-4b27-8e6f-40a470c76d90-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.673582 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.673554 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjjbf\" (UniqueName: \"kubernetes.io/projected/8b23543c-75ef-4b27-8e6f-40a470c76d90-kube-api-access-wjjbf\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.673794 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.673763 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b23543c-75ef-4b27-8e6f-40a470c76d90-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fxrs2\" (UID: \"8b23543c-75ef-4b27-8e6f-40a470c76d90\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.770722 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.770644 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:44.951559 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:44.951486 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2"] Jul 06 14:29:44.955053 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:29:44.955022 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b23543c_75ef_4b27_8e6f_40a470c76d90.slice/crio-d201ec5128131ba4f74d89ec836b7d140014381b8df88abcc65eeda717cd99a0 WatchSource:0}: Error finding container d201ec5128131ba4f74d89ec836b7d140014381b8df88abcc65eeda717cd99a0: Status 404 returned error can't find the container with id d201ec5128131ba4f74d89ec836b7d140014381b8df88abcc65eeda717cd99a0 Jul 06 14:29:45.170835 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:45.170725 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" event={"ID":"8b23543c-75ef-4b27-8e6f-40a470c76d90","Type":"ContainerStarted","Data":"d201ec5128131ba4f74d89ec836b7d140014381b8df88abcc65eeda717cd99a0"} Jul 06 14:29:46.163713 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:46.163654 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:29:46.163805 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:46.163754 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:29:46.164455 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:46.164428 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:29:46.164521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:46.164481 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:29:47.181662 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.181623 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" event={"ID":"6eb69342-99d1-44ca-ae13-077e565f71c9","Type":"ContainerStarted","Data":"71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a"} Jul 06 14:29:47.182145 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.181675 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:47.183356 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.183308 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" event={"ID":"8b23543c-75ef-4b27-8e6f-40a470c76d90","Type":"ContainerStarted","Data":"020b3a4a6e1649d72f4c8cdb83e168195d856e757a760ad357ed7710fbf6cfde"} Jul 06 14:29:47.183501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.183479 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:47.183855 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.183829 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-4dqgm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:29:47.183966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.183888 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" podUID="6eb69342-99d1-44ca-ae13-077e565f71c9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:29:47.184744 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.184724 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-fxrs2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:29:47.184815 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.184766 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" podUID="8b23543c-75ef-4b27-8e6f-40a470c76d90" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:29:47.285599 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.285533 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" podStartSLOduration=2.5533766719999997 podStartE2EDuration="5.285511447s" totalImagesPullingTime="2.732134775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:47.282052356 +0000 UTC m=+322.824739895" watchObservedRunningTime="2026-07-06 14:29:47.285511447 +0000 UTC m=+322.828198993" Jul 06 14:29:47.355337 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:47.355281 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" podStartSLOduration=2.148150488 podStartE2EDuration="3.355265278s" totalImagesPullingTime="1.20711479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:47.354644541 +0000 UTC m=+322.897332079" watchObservedRunningTime="2026-07-06 14:29:47.355265278 +0000 UTC m=+322.897952815" Jul 06 14:29:48.189173 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:48.189128 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-4dqgm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:29:48.189173 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:48.189166 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-fxrs2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:29:48.189553 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:48.189201 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" podUID="8b23543c-75ef-4b27-8e6f-40a470c76d90" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:29:48.189553 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:48.189202 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" podUID="6eb69342-99d1-44ca-ae13-077e565f71c9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:29:51.189708 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.189678 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fxrs2" Jul 06 14:29:51.190139 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.189742 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:51.297472 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.297440 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm"] Jul 06 14:29:51.297700 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.297655 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" podUID="6eb69342-99d1-44ca-ae13-077e565f71c9" containerName="discovery" containerID="cri-o://71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a" gracePeriod=30 Jul 06 14:29:51.546963 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.546938 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:51.613913 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.613887 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6eb69342-99d1-44ca-ae13-077e565f71c9-local-certs\") pod \"6eb69342-99d1-44ca-ae13-077e565f71c9\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " Jul 06 14:29:51.613913 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.613919 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-kubeconfig\") pod \"6eb69342-99d1-44ca-ae13-077e565f71c9\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " Jul 06 14:29:51.614108 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.613940 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-dns-cert\") pod \"6eb69342-99d1-44ca-ae13-077e565f71c9\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " Jul 06 14:29:51.614108 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.613971 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5wpb\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-kube-api-access-g5wpb\") pod \"6eb69342-99d1-44ca-ae13-077e565f71c9\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " Jul 06 14:29:51.614108 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.614015 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-cacerts\") pod \"6eb69342-99d1-44ca-ae13-077e565f71c9\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " Jul 06 14:29:51.614108 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.614072 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-token\") pod \"6eb69342-99d1-44ca-ae13-077e565f71c9\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " Jul 06 14:29:51.614290 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.614132 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-ca-configmap\") pod \"6eb69342-99d1-44ca-ae13-077e565f71c9\" (UID: \"6eb69342-99d1-44ca-ae13-077e565f71c9\") " Jul 06 14:29:51.614800 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.614680 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6eb69342-99d1-44ca-ae13-077e565f71c9" (UID: "6eb69342-99d1-44ca-ae13-077e565f71c9"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:29:51.616585 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.616531 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "6eb69342-99d1-44ca-ae13-077e565f71c9" (UID: "6eb69342-99d1-44ca-ae13-077e565f71c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:29:51.616585 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.616559 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6eb69342-99d1-44ca-ae13-077e565f71c9" (UID: "6eb69342-99d1-44ca-ae13-077e565f71c9"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:29:51.616749 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.616708 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6eb69342-99d1-44ca-ae13-077e565f71c9-local-certs" (OuterVolumeSpecName: "local-certs") pod "6eb69342-99d1-44ca-ae13-077e565f71c9" (UID: "6eb69342-99d1-44ca-ae13-077e565f71c9"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:29:51.616749 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.616745 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-cacerts" (OuterVolumeSpecName: "cacerts") pod "6eb69342-99d1-44ca-ae13-077e565f71c9" (UID: "6eb69342-99d1-44ca-ae13-077e565f71c9"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:29:51.616923 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.616908 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6eb69342-99d1-44ca-ae13-077e565f71c9" (UID: "6eb69342-99d1-44ca-ae13-077e565f71c9"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:29:51.616966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.616925 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-kube-api-access-g5wpb" (OuterVolumeSpecName: "kube-api-access-g5wpb") pod "6eb69342-99d1-44ca-ae13-077e565f71c9" (UID: "6eb69342-99d1-44ca-ae13-077e565f71c9"). InnerVolumeSpecName "kube-api-access-g5wpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:29:51.714704 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.714618 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-token\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:29:51.714704 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.714647 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-ca-configmap\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:29:51.714704 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.714659 2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6eb69342-99d1-44ca-ae13-077e565f71c9-local-certs\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:29:51.714704 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.714670 2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-kubeconfig\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:29:51.714704 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.714678 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-istio-csr-dns-cert\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:29:51.714704 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.714686 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5wpb\" (UniqueName: \"kubernetes.io/projected/6eb69342-99d1-44ca-ae13-077e565f71c9-kube-api-access-g5wpb\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:29:51.714704 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:51.714695 2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6eb69342-99d1-44ca-ae13-077e565f71c9-cacerts\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:29:52.204336 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.204300 2565 generic.go:356] "Generic (PLEG): container finished" podID="6eb69342-99d1-44ca-ae13-077e565f71c9" containerID="71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a" exitCode=0 Jul 06 14:29:52.204726 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.204347 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" Jul 06 14:29:52.204726 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.204345 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" event={"ID":"6eb69342-99d1-44ca-ae13-077e565f71c9","Type":"ContainerDied","Data":"71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a"} Jul 06 14:29:52.204726 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.204453 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm" event={"ID":"6eb69342-99d1-44ca-ae13-077e565f71c9","Type":"ContainerDied","Data":"95bd9b1731d2ba81c92c5d8e84e95724a78b3e292f37fd8cb12aecafcac5222a"} Jul 06 14:29:52.204726 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.204479 2565 scope.go:117] "RemoveContainer" containerID="71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a" Jul 06 14:29:52.213271 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.213252 2565 scope.go:117] "RemoveContainer" containerID="71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a" Jul 06 14:29:52.213531 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:29:52.213513 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a\": container with ID starting with 71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a not found: ID does not exist" containerID="71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a" Jul 06 14:29:52.213564 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.213541 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a"} err="failed to get container status \"71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a\": rpc error: code = NotFound desc = could not find container \"71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a\": container with ID starting with 71f52a592981f908f3445c09e88056b53b36d4c01c3ae42ec1ce36bff19d737a not found: ID does not exist" Jul 06 14:29:52.236352 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.236328 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm"] Jul 06 14:29:52.241389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.241362 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4dqgm"] Jul 06 14:29:52.931604 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:29:52.931560 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6eb69342-99d1-44ca-ae13-077e565f71c9" path="/var/lib/kubelet/pods/6eb69342-99d1-44ca-ae13-077e565f71c9/volumes" Jul 06 14:30:37.335425 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.335389 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p2bgp"] Jul 06 14:30:37.335851 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.335675 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eb69342-99d1-44ca-ae13-077e565f71c9" containerName="discovery" Jul 06 14:30:37.335851 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.335686 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eb69342-99d1-44ca-ae13-077e565f71c9" containerName="discovery" Jul 06 14:30:37.335851 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.335736 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6eb69342-99d1-44ca-ae13-077e565f71c9" containerName="discovery" Jul 06 14:30:37.344986 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.344961 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" Jul 06 14:30:37.348223 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.348196 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 14:30:37.348223 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.348220 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 14:30:37.348417 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.348258 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-4m22z\"" Jul 06 14:30:37.353293 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.353265 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p2bgp"] Jul 06 14:30:37.389079 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.389046 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8dbq\" (UniqueName: \"kubernetes.io/projected/4a6ca335-1070-457d-afba-e2bf43ca7be5-kube-api-access-b8dbq\") pod \"kuadrant-operator-catalog-p2bgp\" (UID: \"4a6ca335-1070-457d-afba-e2bf43ca7be5\") " pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" Jul 06 14:30:37.490159 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.490122 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8dbq\" (UniqueName: \"kubernetes.io/projected/4a6ca335-1070-457d-afba-e2bf43ca7be5-kube-api-access-b8dbq\") pod \"kuadrant-operator-catalog-p2bgp\" (UID: \"4a6ca335-1070-457d-afba-e2bf43ca7be5\") " pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" Jul 06 14:30:37.499445 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.499420 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8dbq\" (UniqueName: \"kubernetes.io/projected/4a6ca335-1070-457d-afba-e2bf43ca7be5-kube-api-access-b8dbq\") pod \"kuadrant-operator-catalog-p2bgp\" (UID: \"4a6ca335-1070-457d-afba-e2bf43ca7be5\") " pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" Jul 06 14:30:37.655389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.655293 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" Jul 06 14:30:37.697662 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.697625 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p2bgp"] Jul 06 14:30:37.802419 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.802389 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p2bgp"] Jul 06 14:30:37.805332 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:30:37.805303 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a6ca335_1070_457d_afba_e2bf43ca7be5.slice/crio-f479000201279e1676c4b333aceb6d065000c037f5e2c10786df650642b1631f WatchSource:0}: Error finding container f479000201279e1676c4b333aceb6d065000c037f5e2c10786df650642b1631f: Status 404 returned error can't find the container with id f479000201279e1676c4b333aceb6d065000c037f5e2c10786df650642b1631f Jul 06 14:30:37.910804 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.910708 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdb9x"] Jul 06 14:30:37.915620 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.915602 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:30:37.924123 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.924100 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdb9x"] Jul 06 14:30:37.994588 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:37.994558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57cc8\" (UniqueName: \"kubernetes.io/projected/a1d0f677-028c-4de7-ad27-8c7dc3496ed2-kube-api-access-57cc8\") pod \"kuadrant-operator-catalog-jdb9x\" (UID: \"a1d0f677-028c-4de7-ad27-8c7dc3496ed2\") " pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:30:38.095632 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:38.095595 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57cc8\" (UniqueName: \"kubernetes.io/projected/a1d0f677-028c-4de7-ad27-8c7dc3496ed2-kube-api-access-57cc8\") pod \"kuadrant-operator-catalog-jdb9x\" (UID: \"a1d0f677-028c-4de7-ad27-8c7dc3496ed2\") " pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:30:38.104510 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:38.104488 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57cc8\" (UniqueName: \"kubernetes.io/projected/a1d0f677-028c-4de7-ad27-8c7dc3496ed2-kube-api-access-57cc8\") pod \"kuadrant-operator-catalog-jdb9x\" (UID: \"a1d0f677-028c-4de7-ad27-8c7dc3496ed2\") " pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:30:38.225845 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:38.225773 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:30:38.365112 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:38.365079 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdb9x"] Jul 06 14:30:38.368654 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:30:38.368628 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d0f677_028c_4de7_ad27_8c7dc3496ed2.slice/crio-b7e5161f3dbc6218e072b3395b925cf25c0eea0a85a95e969696ad7874c9b78e WatchSource:0}: Error finding container b7e5161f3dbc6218e072b3395b925cf25c0eea0a85a95e969696ad7874c9b78e: Status 404 returned error can't find the container with id b7e5161f3dbc6218e072b3395b925cf25c0eea0a85a95e969696ad7874c9b78e Jul 06 14:30:38.384667 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:38.384636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" event={"ID":"4a6ca335-1070-457d-afba-e2bf43ca7be5","Type":"ContainerStarted","Data":"f479000201279e1676c4b333aceb6d065000c037f5e2c10786df650642b1631f"} Jul 06 14:30:38.385744 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:38.385720 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" event={"ID":"a1d0f677-028c-4de7-ad27-8c7dc3496ed2","Type":"ContainerStarted","Data":"b7e5161f3dbc6218e072b3395b925cf25c0eea0a85a95e969696ad7874c9b78e"} Jul 06 14:30:41.398895 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.398863 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" event={"ID":"4a6ca335-1070-457d-afba-e2bf43ca7be5","Type":"ContainerStarted","Data":"296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5"} Jul 06 14:30:41.399378 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.398911 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" podUID="4a6ca335-1070-457d-afba-e2bf43ca7be5" containerName="registry-server" containerID="cri-o://296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5" gracePeriod=2 Jul 06 14:30:41.400379 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.400356 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" event={"ID":"a1d0f677-028c-4de7-ad27-8c7dc3496ed2","Type":"ContainerStarted","Data":"9be434812a64ff60e7eef778abb759129ee5ef1c80d0a3ed44baf385c4b3b9a8"} Jul 06 14:30:41.416878 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.416835 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" podStartSLOduration=1.799833019 podStartE2EDuration="4.416820421s" totalImagesPullingTime="2.616987402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:41.415680719 +0000 UTC m=+376.958368254" watchObservedRunningTime="2026-07-06 14:30:41.416820421 +0000 UTC m=+376.959507959" Jul 06 14:30:41.437305 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.437259 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" podStartSLOduration=2.380426744 podStartE2EDuration="4.437243284s" totalImagesPullingTime="2.05681654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:41.437015817 +0000 UTC m=+376.979703354" watchObservedRunningTime="2026-07-06 14:30:41.437243284 +0000 UTC m=+376.979930822" Jul 06 14:30:41.653526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.653465 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" Jul 06 14:30:41.723696 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.723660 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8dbq\" (UniqueName: \"kubernetes.io/projected/4a6ca335-1070-457d-afba-e2bf43ca7be5-kube-api-access-b8dbq\") pod \"4a6ca335-1070-457d-afba-e2bf43ca7be5\" (UID: \"4a6ca335-1070-457d-afba-e2bf43ca7be5\") " Jul 06 14:30:41.725927 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.725901 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a6ca335-1070-457d-afba-e2bf43ca7be5-kube-api-access-b8dbq" (OuterVolumeSpecName: "kube-api-access-b8dbq") pod "4a6ca335-1070-457d-afba-e2bf43ca7be5" (UID: "4a6ca335-1070-457d-afba-e2bf43ca7be5"). InnerVolumeSpecName "kube-api-access-b8dbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:41.824774 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:41.824735 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8dbq\" (UniqueName: \"kubernetes.io/projected/4a6ca335-1070-457d-afba-e2bf43ca7be5-kube-api-access-b8dbq\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:30:42.405127 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.405093 2565 generic.go:356] "Generic (PLEG): container finished" podID="4a6ca335-1070-457d-afba-e2bf43ca7be5" containerID="296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5" exitCode=0 Jul 06 14:30:42.405592 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.405141 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" Jul 06 14:30:42.405592 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.405182 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" event={"ID":"4a6ca335-1070-457d-afba-e2bf43ca7be5","Type":"ContainerDied","Data":"296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5"} Jul 06 14:30:42.405592 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.405256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p2bgp" event={"ID":"4a6ca335-1070-457d-afba-e2bf43ca7be5","Type":"ContainerDied","Data":"f479000201279e1676c4b333aceb6d065000c037f5e2c10786df650642b1631f"} Jul 06 14:30:42.405592 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.405289 2565 scope.go:117] "RemoveContainer" containerID="296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5" Jul 06 14:30:42.414125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.414109 2565 scope.go:117] "RemoveContainer" containerID="296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5" Jul 06 14:30:42.414379 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:30:42.414346 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5\": container with ID starting with 296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5 not found: ID does not exist" containerID="296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5" Jul 06 14:30:42.414425 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.414386 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5"} err="failed to get container status \"296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5\": rpc error: code = NotFound desc = could not find container \"296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5\": container with ID starting with 296d62e7066df063e9cec5dd7fdea2f0265ab1d5410bbb628a7328f58c45b5f5 not found: ID does not exist" Jul 06 14:30:42.432132 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.432096 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p2bgp"] Jul 06 14:30:42.436206 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.436179 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p2bgp"] Jul 06 14:30:42.932250 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:42.932217 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a6ca335-1070-457d-afba-e2bf43ca7be5" path="/var/lib/kubelet/pods/4a6ca335-1070-457d-afba-e2bf43ca7be5/volumes" Jul 06 14:30:48.226028 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:48.225995 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:30:48.226429 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:48.226048 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:30:48.248519 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:48.248490 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:30:48.458216 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:30:48.458185 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jdb9x" Jul 06 14:31:10.381598 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.381555 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp"] Jul 06 14:31:10.382207 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.382186 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a6ca335-1070-457d-afba-e2bf43ca7be5" containerName="registry-server" Jul 06 14:31:10.382289 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.382212 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a6ca335-1070-457d-afba-e2bf43ca7be5" containerName="registry-server" Jul 06 14:31:10.382382 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.382369 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a6ca335-1070-457d-afba-e2bf43ca7be5" containerName="registry-server" Jul 06 14:31:10.386515 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.386492 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" Jul 06 14:31:10.389112 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.389090 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tm8h5\"" Jul 06 14:31:10.394360 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.394338 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp"] Jul 06 14:31:10.442630 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.442586 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgvzs\" (UniqueName: \"kubernetes.io/projected/5e5d2fe0-7cff-4282-8b00-5d482db88099-kube-api-access-qgvzs\") pod \"authorino-operator-75cc8d5cb7-2cnhp\" (UID: \"5e5d2fe0-7cff-4282-8b00-5d482db88099\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" Jul 06 14:31:10.543711 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.543671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzs\" (UniqueName: \"kubernetes.io/projected/5e5d2fe0-7cff-4282-8b00-5d482db88099-kube-api-access-qgvzs\") pod \"authorino-operator-75cc8d5cb7-2cnhp\" (UID: \"5e5d2fe0-7cff-4282-8b00-5d482db88099\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" Jul 06 14:31:10.562935 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.562908 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgvzs\" (UniqueName: \"kubernetes.io/projected/5e5d2fe0-7cff-4282-8b00-5d482db88099-kube-api-access-qgvzs\") pod \"authorino-operator-75cc8d5cb7-2cnhp\" (UID: \"5e5d2fe0-7cff-4282-8b00-5d482db88099\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" Jul 06 14:31:10.697002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.696965 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" Jul 06 14:31:10.827850 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:10.827738 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp"] Jul 06 14:31:10.830162 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:10.830132 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e5d2fe0_7cff_4282_8b00_5d482db88099.slice/crio-be69a403c19f078f5296af1c4f8634df38ec31e6bf6168b01a325e02db71a10f WatchSource:0}: Error finding container be69a403c19f078f5296af1c4f8634df38ec31e6bf6168b01a325e02db71a10f: Status 404 returned error can't find the container with id be69a403c19f078f5296af1c4f8634df38ec31e6bf6168b01a325e02db71a10f Jul 06 14:31:11.530499 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:11.530419 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" event={"ID":"5e5d2fe0-7cff-4282-8b00-5d482db88099","Type":"ContainerStarted","Data":"be69a403c19f078f5296af1c4f8634df38ec31e6bf6168b01a325e02db71a10f"} Jul 06 14:31:13.538945 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:13.538910 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" event={"ID":"5e5d2fe0-7cff-4282-8b00-5d482db88099","Type":"ContainerStarted","Data":"f2fb11779e59d559b6c0e9bad23b8581dff98853a1a94c7f8aad7cfc4864744c"} Jul 06 14:31:13.539315 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:13.539003 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" Jul 06 14:31:13.562716 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:13.562663 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" podStartSLOduration=1.51939043 podStartE2EDuration="3.56264636s" totalImagesPullingTime="2.04325593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:13.560254954 +0000 UTC m=+409.102942491" watchObservedRunningTime="2026-07-06 14:31:13.56264636 +0000 UTC m=+409.105333898" Jul 06 14:31:14.733634 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.733600 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9"] Jul 06 14:31:14.736937 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.736918 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:14.740132 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.740111 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dpjsc\"" Jul 06 14:31:14.752670 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.752645 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9"] Jul 06 14:31:14.782996 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.782962 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" (UID: \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:14.783148 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.783011 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kthw\" (UniqueName: \"kubernetes.io/projected/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-kube-api-access-8kthw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" (UID: \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:14.884331 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.884292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kthw\" (UniqueName: \"kubernetes.io/projected/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-kube-api-access-8kthw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" (UID: \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:14.884501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.884383 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" (UID: \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:14.884773 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.884755 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" (UID: \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:14.900535 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:14.900506 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kthw\" (UniqueName: \"kubernetes.io/projected/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-kube-api-access-8kthw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" (UID: \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:15.047582 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:15.047478 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:15.188583 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:15.188551 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9"] Jul 06 14:31:15.190908 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:15.190881 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9b4f33_3ded_464c_bf61_7e8c43cd4b05.slice/crio-a10bf8be5a35a3fac9c36b750130ac7b5acdfd2088045c366131e91c3e41c837 WatchSource:0}: Error finding container a10bf8be5a35a3fac9c36b750130ac7b5acdfd2088045c366131e91c3e41c837: Status 404 returned error can't find the container with id a10bf8be5a35a3fac9c36b750130ac7b5acdfd2088045c366131e91c3e41c837 Jul 06 14:31:15.548043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:15.548008 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" event={"ID":"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05","Type":"ContainerStarted","Data":"a10bf8be5a35a3fac9c36b750130ac7b5acdfd2088045c366131e91c3e41c837"} Jul 06 14:31:16.870556 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:16.870515 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v"] Jul 06 14:31:16.874174 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:16.874143 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" Jul 06 14:31:16.877372 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:16.877343 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 14:31:16.877516 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:16.877342 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rptqf\"" Jul 06 14:31:16.884712 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:16.884687 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v"] Jul 06 14:31:17.004242 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:17.004211 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7k8s\" (UniqueName: \"kubernetes.io/projected/32b889b1-0ea1-4cf9-a902-31e64a7f8e95-kube-api-access-g7k8s\") pod \"dns-operator-controller-manager-864dd78c4c-84q7v\" (UID: \"32b889b1-0ea1-4cf9-a902-31e64a7f8e95\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" Jul 06 14:31:17.105611 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:17.105565 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7k8s\" (UniqueName: \"kubernetes.io/projected/32b889b1-0ea1-4cf9-a902-31e64a7f8e95-kube-api-access-g7k8s\") pod \"dns-operator-controller-manager-864dd78c4c-84q7v\" (UID: \"32b889b1-0ea1-4cf9-a902-31e64a7f8e95\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" Jul 06 14:31:17.117632 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:17.117600 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7k8s\" (UniqueName: \"kubernetes.io/projected/32b889b1-0ea1-4cf9-a902-31e64a7f8e95-kube-api-access-g7k8s\") pod \"dns-operator-controller-manager-864dd78c4c-84q7v\" (UID: \"32b889b1-0ea1-4cf9-a902-31e64a7f8e95\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" Jul 06 14:31:17.187628 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:17.187589 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" Jul 06 14:31:17.346462 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:17.346429 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v"] Jul 06 14:31:17.351069 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:17.351041 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b889b1_0ea1_4cf9_a902_31e64a7f8e95.slice/crio-9a511b18cb489087435ad3d6b8566354d213a033715d59d3c9cdd6bb80739f74 WatchSource:0}: Error finding container 9a511b18cb489087435ad3d6b8566354d213a033715d59d3c9cdd6bb80739f74: Status 404 returned error can't find the container with id 9a511b18cb489087435ad3d6b8566354d213a033715d59d3c9cdd6bb80739f74 Jul 06 14:31:17.558209 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:17.558119 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" event={"ID":"32b889b1-0ea1-4cf9-a902-31e64a7f8e95","Type":"ContainerStarted","Data":"9a511b18cb489087435ad3d6b8566354d213a033715d59d3c9cdd6bb80739f74"} Jul 06 14:31:19.496189 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.496155 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l"] Jul 06 14:31:19.499458 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.499440 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:19.502301 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.502277 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kgb7r\"" Jul 06 14:31:19.513594 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.513551 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l"] Jul 06 14:31:19.570611 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.570547 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" event={"ID":"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05","Type":"ContainerStarted","Data":"fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f"} Jul 06 14:31:19.570833 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.570698 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:19.590423 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.590202 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" podStartSLOduration=1.978717954 podStartE2EDuration="5.590180562s" totalImagesPullingTime="3.611462608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:19.589289822 +0000 UTC m=+415.131977356" watchObservedRunningTime="2026-07-06 14:31:19.590180562 +0000 UTC m=+415.132868101" Jul 06 14:31:19.627712 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.627679 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h594m\" (UniqueName: \"kubernetes.io/projected/2a642f31-c5c9-49b8-98fa-6b5fd4038546-kube-api-access-h594m\") pod \"limitador-operator-controller-manager-859d7b78bd-49c7l\" (UID: \"2a642f31-c5c9-49b8-98fa-6b5fd4038546\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:19.728955 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.728918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h594m\" (UniqueName: \"kubernetes.io/projected/2a642f31-c5c9-49b8-98fa-6b5fd4038546-kube-api-access-h594m\") pod \"limitador-operator-controller-manager-859d7b78bd-49c7l\" (UID: \"2a642f31-c5c9-49b8-98fa-6b5fd4038546\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:19.740471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.740436 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h594m\" (UniqueName: \"kubernetes.io/projected/2a642f31-c5c9-49b8-98fa-6b5fd4038546-kube-api-access-h594m\") pod \"limitador-operator-controller-manager-859d7b78bd-49c7l\" (UID: \"2a642f31-c5c9-49b8-98fa-6b5fd4038546\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:19.812843 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:19.812755 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:20.220478 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:20.220307 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l"] Jul 06 14:31:20.222872 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:20.222745 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a642f31_c5c9_49b8_98fa_6b5fd4038546.slice/crio-e1d14386bfa32a990d9dba2770b96219c6f86307c72e72d5e0a047e0abae3ccc WatchSource:0}: Error finding container e1d14386bfa32a990d9dba2770b96219c6f86307c72e72d5e0a047e0abae3ccc: Status 404 returned error can't find the container with id e1d14386bfa32a990d9dba2770b96219c6f86307c72e72d5e0a047e0abae3ccc Jul 06 14:31:20.576976 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:20.576942 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" event={"ID":"32b889b1-0ea1-4cf9-a902-31e64a7f8e95","Type":"ContainerStarted","Data":"a4610f322ebec920dd327469faa561ad7d61e242846cfd049e0643966b2b7f3e"} Jul 06 14:31:20.577423 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:20.577036 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" Jul 06 14:31:20.578223 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:20.578201 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" event={"ID":"2a642f31-c5c9-49b8-98fa-6b5fd4038546","Type":"ContainerStarted","Data":"e1d14386bfa32a990d9dba2770b96219c6f86307c72e72d5e0a047e0abae3ccc"} Jul 06 14:31:20.599099 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:20.599047 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" podStartSLOduration=1.81815856 podStartE2EDuration="4.599032529s" totalImagesPullingTime="2.780873969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:20.597032255 +0000 UTC m=+416.139719806" watchObservedRunningTime="2026-07-06 14:31:20.599032529 +0000 UTC m=+416.141720098" Jul 06 14:31:21.729685 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.729648 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq"] Jul 06 14:31:21.733456 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.733433 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.736368 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.736348 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c89dr\"" Jul 06 14:31:21.736618 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.736604 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 14:31:21.736885 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.736871 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 14:31:21.743000 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.742978 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq"] Jul 06 14:31:21.847270 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.847187 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms5lq\" (UniqueName: \"kubernetes.io/projected/321780b5-a07c-47f1-8fcb-913731118317-kube-api-access-ms5lq\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.847270 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.847223 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/321780b5-a07c-47f1-8fcb-913731118317-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.847270 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.847250 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/321780b5-a07c-47f1-8fcb-913731118317-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.947934 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.947900 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms5lq\" (UniqueName: \"kubernetes.io/projected/321780b5-a07c-47f1-8fcb-913731118317-kube-api-access-ms5lq\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.947934 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.947936 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/321780b5-a07c-47f1-8fcb-913731118317-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.948198 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.947963 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/321780b5-a07c-47f1-8fcb-913731118317-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.948676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.948655 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/321780b5-a07c-47f1-8fcb-913731118317-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.950573 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.950554 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/321780b5-a07c-47f1-8fcb-913731118317-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:21.961662 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:21.961637 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms5lq\" (UniqueName: \"kubernetes.io/projected/321780b5-a07c-47f1-8fcb-913731118317-kube-api-access-ms5lq\") pod \"kuadrant-console-plugin-66b5774985-gcprq\" (UID: \"321780b5-a07c-47f1-8fcb-913731118317\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:22.043218 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:22.043176 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" Jul 06 14:31:22.172242 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:22.172207 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq"] Jul 06 14:31:22.174592 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:22.174567 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod321780b5_a07c_47f1_8fcb_913731118317.slice/crio-54c07d10afee16198a6c2159414169d5a86cb3adb935d090561d1a155b0ff267 WatchSource:0}: Error finding container 54c07d10afee16198a6c2159414169d5a86cb3adb935d090561d1a155b0ff267: Status 404 returned error can't find the container with id 54c07d10afee16198a6c2159414169d5a86cb3adb935d090561d1a155b0ff267 Jul 06 14:31:22.587828 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:22.587773 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" event={"ID":"321780b5-a07c-47f1-8fcb-913731118317","Type":"ContainerStarted","Data":"54c07d10afee16198a6c2159414169d5a86cb3adb935d090561d1a155b0ff267"} Jul 06 14:31:22.589185 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:22.589159 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" event={"ID":"2a642f31-c5c9-49b8-98fa-6b5fd4038546","Type":"ContainerStarted","Data":"1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58"} Jul 06 14:31:22.589320 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:22.589303 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:22.618052 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:22.618007 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" podStartSLOduration=2.264534371 podStartE2EDuration="3.617992591s" totalImagesPullingTime="1.35345822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:22.615453723 +0000 UTC m=+418.158141260" watchObservedRunningTime="2026-07-06 14:31:22.617992591 +0000 UTC m=+418.160680129" Jul 06 14:31:24.545975 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:24.545939 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2cnhp" Jul 06 14:31:30.580913 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:30.580884 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:31.586498 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:31.586465 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84q7v" Jul 06 14:31:31.926531 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:31.926431 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8"] Jul 06 14:31:31.947918 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:31.947888 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:31.954329 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:31.954301 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8"] Jul 06 14:31:32.039091 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.039046 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98e3719-6240-4f28-a33c-bf77fe343803-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" (UID: \"e98e3719-6240-4f28-a33c-bf77fe343803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:32.039251 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.039143 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms6mj\" (UniqueName: \"kubernetes.io/projected/e98e3719-6240-4f28-a33c-bf77fe343803-kube-api-access-ms6mj\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" (UID: \"e98e3719-6240-4f28-a33c-bf77fe343803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:32.140419 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.140379 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98e3719-6240-4f28-a33c-bf77fe343803-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" (UID: \"e98e3719-6240-4f28-a33c-bf77fe343803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:32.140599 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.140452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms6mj\" (UniqueName: \"kubernetes.io/projected/e98e3719-6240-4f28-a33c-bf77fe343803-kube-api-access-ms6mj\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" (UID: \"e98e3719-6240-4f28-a33c-bf77fe343803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:32.140838 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.140777 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98e3719-6240-4f28-a33c-bf77fe343803-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" (UID: \"e98e3719-6240-4f28-a33c-bf77fe343803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:32.151604 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.151572 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms6mj\" (UniqueName: \"kubernetes.io/projected/e98e3719-6240-4f28-a33c-bf77fe343803-kube-api-access-ms6mj\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" (UID: \"e98e3719-6240-4f28-a33c-bf77fe343803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:32.261024 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.260990 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:32.463399 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.463360 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8"] Jul 06 14:31:32.469243 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.469211 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8"] Jul 06 14:31:32.483395 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.483353 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9"] Jul 06 14:31:32.483581 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.483554 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" containerName="manager" containerID="cri-o://fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f" gracePeriod=2 Jul 06 14:31:32.489481 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.489448 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4"] Jul 06 14:31:32.499341 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.498899 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9"] Jul 06 14:31:32.499341 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.499057 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:32.501388 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.501352 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l"] Jul 06 14:31:32.503776 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.502289 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" containerName="manager" containerID="cri-o://1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58" gracePeriod=2 Jul 06 14:31:32.504096 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.504067 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:32.505841 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.505820 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4"] Jul 06 14:31:32.508000 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.507975 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89"] Jul 06 14:31:32.508403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.508381 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" containerName="manager" Jul 06 14:31:32.508403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.508401 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" containerName="manager" Jul 06 14:31:32.508615 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.508493 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" containerName="manager" Jul 06 14:31:32.519618 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.519535 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l"] Jul 06 14:31:32.519739 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.519662 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:32.529356 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.529331 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62"] Jul 06 14:31:32.529937 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.529912 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" containerName="manager" Jul 06 14:31:32.529937 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.529937 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" containerName="manager" Jul 06 14:31:32.530116 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.530002 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" containerName="manager" Jul 06 14:31:32.532989 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.532966 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89"] Jul 06 14:31:32.533098 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.533071 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" Jul 06 14:31:32.533954 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:32.533921 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:32.536631 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:32.536455 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:32.538950 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:32.538917 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49c7l\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:32.544378 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.544355 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62"] Jul 06 14:31:32.643532 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.643499 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrrfq\" (UniqueName: \"kubernetes.io/projected/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-kube-api-access-jrrfq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppr89\" (UID: \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:32.643927 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.643591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppr89\" (UID: \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:32.643927 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.643617 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsfhs\" (UniqueName: \"kubernetes.io/projected/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-kube-api-access-gsfhs\") pod \"kuadrant-operator-controller-manager-55467d9f7-kxlw4\" (UID: \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:32.643927 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.643695 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vrwx\" (UniqueName: \"kubernetes.io/projected/afe2556f-be20-4740-9ccc-7894305d6442-kube-api-access-2vrwx\") pod \"limitador-operator-controller-manager-859d7b78bd-gfr62\" (UID: \"afe2556f-be20-4740-9ccc-7894305d6442\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" Jul 06 14:31:32.643927 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.643822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-kxlw4\" (UID: \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:32.744831 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.744771 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-kxlw4\" (UID: \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:32.744995 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.744842 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrrfq\" (UniqueName: \"kubernetes.io/projected/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-kube-api-access-jrrfq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppr89\" (UID: \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:32.744995 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.744883 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppr89\" (UID: \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:32.744995 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.744904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfhs\" (UniqueName: \"kubernetes.io/projected/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-kube-api-access-gsfhs\") pod \"kuadrant-operator-controller-manager-55467d9f7-kxlw4\" (UID: \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:32.744995 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.744928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vrwx\" (UniqueName: \"kubernetes.io/projected/afe2556f-be20-4740-9ccc-7894305d6442-kube-api-access-2vrwx\") pod \"limitador-operator-controller-manager-859d7b78bd-gfr62\" (UID: \"afe2556f-be20-4740-9ccc-7894305d6442\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" Jul 06 14:31:32.745249 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.745218 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-kxlw4\" (UID: \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:32.745307 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.745275 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppr89\" (UID: \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:32.756148 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.756122 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrrfq\" (UniqueName: \"kubernetes.io/projected/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-kube-api-access-jrrfq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppr89\" (UID: \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:32.756322 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.756299 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsfhs\" (UniqueName: \"kubernetes.io/projected/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-kube-api-access-gsfhs\") pod \"kuadrant-operator-controller-manager-55467d9f7-kxlw4\" (UID: \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:32.756636 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.756615 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vrwx\" (UniqueName: \"kubernetes.io/projected/afe2556f-be20-4740-9ccc-7894305d6442-kube-api-access-2vrwx\") pod \"limitador-operator-controller-manager-859d7b78bd-gfr62\" (UID: \"afe2556f-be20-4740-9ccc-7894305d6442\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" Jul 06 14:31:32.931087 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.930999 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:32.939156 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.939122 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:32.947019 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:32.946985 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" Jul 06 14:31:44.500909 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.498637 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89"] Jul 06 14:31:44.503903 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:44.503872 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5dfb0f5_e479_41c9_bdea_a7b000fc5d6c.slice/crio-36b2b64793c028bc52a3b4d6620e79f8c7bac4a21b7dc28c6e892258be2e328f WatchSource:0}: Error finding container 36b2b64793c028bc52a3b4d6620e79f8c7bac4a21b7dc28c6e892258be2e328f: Status 404 returned error can't find the container with id 36b2b64793c028bc52a3b4d6620e79f8c7bac4a21b7dc28c6e892258be2e328f Jul 06 14:31:44.518215 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.518194 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:44.521096 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.521072 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49c7l\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:44.546888 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.546869 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:44.549827 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.549773 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49c7l\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:44.556504 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.556479 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:44.560775 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.560756 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h594m\" (UniqueName: \"kubernetes.io/projected/2a642f31-c5c9-49b8-98fa-6b5fd4038546-kube-api-access-h594m\") pod \"2a642f31-c5c9-49b8-98fa-6b5fd4038546\" (UID: \"2a642f31-c5c9-49b8-98fa-6b5fd4038546\") " Jul 06 14:31:44.560896 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.560843 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-extensions-socket-volume\") pod \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\" (UID: \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\") " Jul 06 14:31:44.560896 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.560888 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kthw\" (UniqueName: \"kubernetes.io/projected/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-kube-api-access-8kthw\") pod \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\" (UID: \"5c9b4f33-3ded-464c-bf61-7e8c43cd4b05\") " Jul 06 14:31:44.561645 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.561415 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" (UID: "5c9b4f33-3ded-464c-bf61-7e8c43cd4b05"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:31:44.563388 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.563363 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a642f31-c5c9-49b8-98fa-6b5fd4038546-kube-api-access-h594m" (OuterVolumeSpecName: "kube-api-access-h594m") pod "2a642f31-c5c9-49b8-98fa-6b5fd4038546" (UID: "2a642f31-c5c9-49b8-98fa-6b5fd4038546"). InnerVolumeSpecName "kube-api-access-h594m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:31:44.564204 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.564181 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-kube-api-access-8kthw" (OuterVolumeSpecName: "kube-api-access-8kthw") pod "5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" (UID: "5c9b4f33-3ded-464c-bf61-7e8c43cd4b05"). InnerVolumeSpecName "kube-api-access-8kthw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:31:44.662024 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.661993 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h594m\" (UniqueName: \"kubernetes.io/projected/2a642f31-c5c9-49b8-98fa-6b5fd4038546-kube-api-access-h594m\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:31:44.662024 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.662023 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-extensions-socket-volume\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:31:44.662204 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.662033 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kthw\" (UniqueName: \"kubernetes.io/projected/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05-kube-api-access-8kthw\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:31:44.706360 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.706310 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" event={"ID":"321780b5-a07c-47f1-8fcb-913731118317","Type":"ContainerStarted","Data":"ccf5cf49675f022f8666e82e32b97615ba99dc331d662cd067dc6c93eec64db2"} Jul 06 14:31:44.707573 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.707549 2565 generic.go:356] "Generic (PLEG): container finished" podID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" containerID="1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58" exitCode=0 Jul 06 14:31:44.707684 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.707579 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:44.707684 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.707631 2565 scope.go:117] "RemoveContainer" containerID="1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58" Jul 06 14:31:44.711817 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.709192 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49c7l\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:44.711817 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.711740 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:44.718126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.715446 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" event={"ID":"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c","Type":"ContainerStarted","Data":"620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036"} Jul 06 14:31:44.718126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.715553 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" event={"ID":"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c","Type":"ContainerStarted","Data":"36b2b64793c028bc52a3b4d6620e79f8c7bac4a21b7dc28c6e892258be2e328f"} Jul 06 14:31:44.718126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.716173 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:44.719328 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.719301 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62"] Jul 06 14:31:44.721212 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.720745 2565 generic.go:356] "Generic (PLEG): container finished" podID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" containerID="fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f" exitCode=0 Jul 06 14:31:44.721212 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.721165 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:44.722756 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:44.722731 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe2556f_be20_4740_9ccc_7894305d6442.slice/crio-5b0e0e9a2b24f9dbdc832b2c131a945b371ef1d555004f3c711ed5b3149a2daf WatchSource:0}: Error finding container 5b0e0e9a2b24f9dbdc832b2c131a945b371ef1d555004f3c711ed5b3149a2daf: Status 404 returned error can't find the container with id 5b0e0e9a2b24f9dbdc832b2c131a945b371ef1d555004f3c711ed5b3149a2daf Jul 06 14:31:44.723232 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.723211 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4"] Jul 06 14:31:44.724091 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:44.724071 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ddda26_8bbf_4b0f_b5bd_bb6c149deec9.slice/crio-62e1e6eea5456513b2dc0b8a9a9475379de4f4b5184ee2e7edbf1042f414e85a WatchSource:0}: Error finding container 62e1e6eea5456513b2dc0b8a9a9475379de4f4b5184ee2e7edbf1042f414e85a: Status 404 returned error can't find the container with id 62e1e6eea5456513b2dc0b8a9a9475379de4f4b5184ee2e7edbf1042f414e85a Jul 06 14:31:44.727546 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.727521 2565 scope.go:117] "RemoveContainer" containerID="1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58" Jul 06 14:31:44.727842 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.727812 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58\": container with ID starting with 1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58 not found: ID does not exist" containerID="1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58" Jul 06 14:31:44.727936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.727850 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58"} err="failed to get container status \"1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58\": rpc error: code = NotFound desc = could not find container \"1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58\": container with ID starting with 1e9044650d8025f34a9e0a159cea37cc232e32c69f9c36dccf82ec983c8fef58 not found: ID does not exist" Jul 06 14:31:44.727936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.727871 2565 scope.go:117] "RemoveContainer" containerID="fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f" Jul 06 14:31:44.736066 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.736021 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gcprq" podStartSLOduration=1.408723472 podStartE2EDuration="23.736001348s" totalImagesPullingTime="22.327277876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:44.734614214 +0000 UTC m=+440.277301754" watchObservedRunningTime="2026-07-06 14:31:44.736001348 +0000 UTC m=+440.278688889" Jul 06 14:31:44.736139 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.736077 2565 scope.go:117] "RemoveContainer" containerID="fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f" Jul 06 14:31:44.736368 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.736347 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f\": container with ID starting with fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f not found: ID does not exist" containerID="fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f" Jul 06 14:31:44.736411 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.736377 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f"} err="failed to get container status \"fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f\": rpc error: code = NotFound desc = could not find container \"fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f\": container with ID starting with fae4002de7a8c1749078a62b53efef77f7bdfd11b9b0721a9cff7ed4c0d15d6f not found: ID does not exist" Jul 06 14:31:44.761501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.761448 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" podStartSLOduration=12.761424586 podStartE2EDuration="12.761424586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:44.760259757 +0000 UTC m=+440.302947296" watchObservedRunningTime="2026-07-06 14:31:44.761424586 +0000 UTC m=+440.304112126" Jul 06 14:31:44.766062 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.766025 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49c7l\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:44.768749 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.768719 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:44.931277 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.931245 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49c7l\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49c7l" Jul 06 14:31:44.932385 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.932363 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a642f31-c5c9-49b8-98fa-6b5fd4038546" path="/var/lib/kubelet/pods/2a642f31-c5c9-49b8-98fa-6b5fd4038546/volumes" Jul 06 14:31:44.932686 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:44.932674 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" path="/var/lib/kubelet/pods/5c9b4f33-3ded-464c-bf61-7e8c43cd4b05/volumes" Jul 06 14:31:44.933539 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:44.933517 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-84wn9\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="5c9b4f33-3ded-464c-bf61-7e8c43cd4b05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-84wn9" Jul 06 14:31:45.726373 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:45.726337 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" event={"ID":"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9","Type":"ContainerStarted","Data":"4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6"} Jul 06 14:31:45.726844 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:45.726402 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" event={"ID":"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9","Type":"ContainerStarted","Data":"62e1e6eea5456513b2dc0b8a9a9475379de4f4b5184ee2e7edbf1042f414e85a"} Jul 06 14:31:45.726844 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:45.726435 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:45.728600 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:45.728573 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" event={"ID":"afe2556f-be20-4740-9ccc-7894305d6442","Type":"ContainerStarted","Data":"5b9ece9573a941b87875016c264486fd9f38bd10c8f28fe5877dc7958040789c"} Jul 06 14:31:45.728712 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:45.728621 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" Jul 06 14:31:45.728712 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:45.728645 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" event={"ID":"afe2556f-be20-4740-9ccc-7894305d6442","Type":"ContainerStarted","Data":"5b0e0e9a2b24f9dbdc832b2c131a945b371ef1d555004f3c711ed5b3149a2daf"} Jul 06 14:31:45.750714 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:45.750669 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" podStartSLOduration=13.750655899 podStartE2EDuration="13.750655899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:45.748444687 +0000 UTC m=+441.291132236" watchObservedRunningTime="2026-07-06 14:31:45.750655899 +0000 UTC m=+441.293343437" Jul 06 14:31:45.769605 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:45.769557 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" podStartSLOduration=13.769541795 podStartE2EDuration="13.769541795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:45.767740048 +0000 UTC m=+441.310427597" watchObservedRunningTime="2026-07-06 14:31:45.769541795 +0000 UTC m=+441.312229333" Jul 06 14:31:46.848132 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:46.848089 2565 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-m8dq8_kuadrant-system_e98e3719-6240-4f28-a33c-bf77fe343803_0(6ea14d294fcabfd349bedd528e0c22db66203124f83fb3c36f7cce046a82d9f2): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-m8dq8 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 06 14:31:46.848391 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:46.848159 2565 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-m8dq8_kuadrant-system_e98e3719-6240-4f28-a33c-bf77fe343803_0(6ea14d294fcabfd349bedd528e0c22db66203124f83fb3c36f7cce046a82d9f2): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-m8dq8 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:47.737742 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:47.737715 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:47.740270 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:47.740231 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="e98e3719-6240-4f28-a33c-bf77fe343803" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:47.783916 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:47.783885 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:47.786675 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:47.786647 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="e98e3719-6240-4f28-a33c-bf77fe343803" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:47.891235 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:47.891198 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98e3719-6240-4f28-a33c-bf77fe343803-extensions-socket-volume\") pod \"e98e3719-6240-4f28-a33c-bf77fe343803\" (UID: \"e98e3719-6240-4f28-a33c-bf77fe343803\") " Jul 06 14:31:47.891235 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:47.891230 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ms6mj\" (UniqueName: \"kubernetes.io/projected/e98e3719-6240-4f28-a33c-bf77fe343803-kube-api-access-ms6mj\") pod \"e98e3719-6240-4f28-a33c-bf77fe343803\" (UID: \"e98e3719-6240-4f28-a33c-bf77fe343803\") " Jul 06 14:31:47.891680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:47.891461 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98e3719-6240-4f28-a33c-bf77fe343803-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e98e3719-6240-4f28-a33c-bf77fe343803" (UID: "e98e3719-6240-4f28-a33c-bf77fe343803"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:31:47.893547 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:47.893523 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e98e3719-6240-4f28-a33c-bf77fe343803-kube-api-access-ms6mj" (OuterVolumeSpecName: "kube-api-access-ms6mj") pod "e98e3719-6240-4f28-a33c-bf77fe343803" (UID: "e98e3719-6240-4f28-a33c-bf77fe343803"). InnerVolumeSpecName "kube-api-access-ms6mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:31:47.992262 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:47.992174 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98e3719-6240-4f28-a33c-bf77fe343803-extensions-socket-volume\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:31:47.992262 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:47.992206 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms6mj\" (UniqueName: \"kubernetes.io/projected/e98e3719-6240-4f28-a33c-bf77fe343803-kube-api-access-ms6mj\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:31:48.741478 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:48.741448 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:48.747640 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:48.747601 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="e98e3719-6240-4f28-a33c-bf77fe343803" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:48.750184 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:48.750160 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m8dq8\" is forbidden: User \"system:node:ip-10-0-128-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-205.ec2.internal' and this object" podUID="e98e3719-6240-4f28-a33c-bf77fe343803" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8dq8" Jul 06 14:31:48.932519 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:48.932483 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e98e3719-6240-4f28-a33c-bf77fe343803" path="/var/lib/kubelet/pods/e98e3719-6240-4f28-a33c-bf77fe343803/volumes" Jul 06 14:31:55.732355 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:55.732325 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:56.736281 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:56.736253 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfr62" Jul 06 14:31:56.736769 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:56.736337 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:31:56.805282 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:56.805252 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89"] Jul 06 14:31:56.805504 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:56.805480 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" podUID="c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c" containerName="manager" containerID="cri-o://620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036" gracePeriod=10 Jul 06 14:31:57.054376 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.054350 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:57.075307 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.075282 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn"] Jul 06 14:31:57.075612 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.075600 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c" containerName="manager" Jul 06 14:31:57.075650 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.075614 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c" containerName="manager" Jul 06 14:31:57.075712 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.075702 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c" containerName="manager" Jul 06 14:31:57.077331 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.077315 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.096180 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.096149 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn"] Jul 06 14:31:57.166214 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.166170 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-extensions-socket-volume\") pod \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\" (UID: \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\") " Jul 06 14:31:57.166389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.166244 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrrfq\" (UniqueName: \"kubernetes.io/projected/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-kube-api-access-jrrfq\") pod \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\" (UID: \"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c\") " Jul 06 14:31:57.166389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.166366 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bng4d\" (UniqueName: \"kubernetes.io/projected/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-kube-api-access-bng4d\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8xn\" (UID: \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.166471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.166422 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8xn\" (UID: \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.166574 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.166553 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c" (UID: "c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:31:57.168433 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.168415 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-kube-api-access-jrrfq" (OuterVolumeSpecName: "kube-api-access-jrrfq") pod "c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c" (UID: "c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c"). InnerVolumeSpecName "kube-api-access-jrrfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:31:57.267702 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.267662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bng4d\" (UniqueName: \"kubernetes.io/projected/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-kube-api-access-bng4d\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8xn\" (UID: \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.267906 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.267708 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8xn\" (UID: \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.267906 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.267838 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-extensions-socket-volume\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:31:57.267906 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.267855 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrrfq\" (UniqueName: \"kubernetes.io/projected/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c-kube-api-access-jrrfq\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:31:57.268113 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.268095 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8xn\" (UID: \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.279588 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.279520 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bng4d\" (UniqueName: \"kubernetes.io/projected/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-kube-api-access-bng4d\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8xn\" (UID: \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.387872 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.387825 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.526323 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.526296 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn"] Jul 06 14:31:57.529127 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:31:57.529097 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbbf7dd_d5c2_4516_ac38_74cabf19aa08.slice/crio-8d8b4426f22242f428b954591c3d4085c319cb9bb87f8c57abe0c347b4687249 WatchSource:0}: Error finding container 8d8b4426f22242f428b954591c3d4085c319cb9bb87f8c57abe0c347b4687249: Status 404 returned error can't find the container with id 8d8b4426f22242f428b954591c3d4085c319cb9bb87f8c57abe0c347b4687249 Jul 06 14:31:57.784296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.784203 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" event={"ID":"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08","Type":"ContainerStarted","Data":"8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3"} Jul 06 14:31:57.784296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.784262 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" event={"ID":"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08","Type":"ContainerStarted","Data":"8d8b4426f22242f428b954591c3d4085c319cb9bb87f8c57abe0c347b4687249"} Jul 06 14:31:57.784296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.784293 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:31:57.785414 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.785389 2565 generic.go:356] "Generic (PLEG): container finished" podID="c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c" containerID="620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036" exitCode=0 Jul 06 14:31:57.785478 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.785428 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" Jul 06 14:31:57.785478 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.785435 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" event={"ID":"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c","Type":"ContainerDied","Data":"620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036"} Jul 06 14:31:57.785555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.785479 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89" event={"ID":"c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c","Type":"ContainerDied","Data":"36b2b64793c028bc52a3b4d6620e79f8c7bac4a21b7dc28c6e892258be2e328f"} Jul 06 14:31:57.785555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.785509 2565 scope.go:117] "RemoveContainer" containerID="620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036" Jul 06 14:31:57.794002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.793986 2565 scope.go:117] "RemoveContainer" containerID="620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036" Jul 06 14:31:57.794255 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:31:57.794236 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036\": container with ID starting with 620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036 not found: ID does not exist" containerID="620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036" Jul 06 14:31:57.794309 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.794263 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036"} err="failed to get container status \"620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036\": rpc error: code = NotFound desc = could not find container \"620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036\": container with ID starting with 620b3b14bed3ed2fcc127a503584fa9e40e27e803381c078371f4f546c9e5036 not found: ID does not exist" Jul 06 14:31:57.809594 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.809509 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" podStartSLOduration=0.809492212 podStartE2EDuration="809.492212ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:57.809439478 +0000 UTC m=+453.352127018" watchObservedRunningTime="2026-07-06 14:31:57.809492212 +0000 UTC m=+453.352179755" Jul 06 14:31:57.830100 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.830071 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89"] Jul 06 14:31:57.834618 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:57.834596 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppr89"] Jul 06 14:31:58.932835 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:31:58.932778 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c" path="/var/lib/kubelet/pods/c5dfb0f5-e479-41c9-bdea-a7b000fc5d6c/volumes" Jul 06 14:32:08.793481 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:08.793450 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:32:08.854273 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:08.854237 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4"] Jul 06 14:32:08.854468 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:08.854428 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" podUID="56ddda26-8bbf-4b0f-b5bd-bb6c149deec9" containerName="manager" containerID="cri-o://4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6" gracePeriod=10 Jul 06 14:32:09.105499 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.105473 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:32:09.168446 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.168414 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsfhs\" (UniqueName: \"kubernetes.io/projected/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-kube-api-access-gsfhs\") pod \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\" (UID: \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\") " Jul 06 14:32:09.168607 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.168521 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-extensions-socket-volume\") pod \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\" (UID: \"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9\") " Jul 06 14:32:09.168971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.168948 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "56ddda26-8bbf-4b0f-b5bd-bb6c149deec9" (UID: "56ddda26-8bbf-4b0f-b5bd-bb6c149deec9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:32:09.170732 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.170710 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-kube-api-access-gsfhs" (OuterVolumeSpecName: "kube-api-access-gsfhs") pod "56ddda26-8bbf-4b0f-b5bd-bb6c149deec9" (UID: "56ddda26-8bbf-4b0f-b5bd-bb6c149deec9"). InnerVolumeSpecName "kube-api-access-gsfhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:32:09.269212 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.269176 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-extensions-socket-volume\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:32:09.269212 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.269208 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsfhs\" (UniqueName: \"kubernetes.io/projected/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9-kube-api-access-gsfhs\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:32:09.839561 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.839526 2565 generic.go:356] "Generic (PLEG): container finished" podID="56ddda26-8bbf-4b0f-b5bd-bb6c149deec9" containerID="4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6" exitCode=0 Jul 06 14:32:09.840032 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.839571 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" Jul 06 14:32:09.840032 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.839620 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" event={"ID":"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9","Type":"ContainerDied","Data":"4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6"} Jul 06 14:32:09.840032 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.839672 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4" event={"ID":"56ddda26-8bbf-4b0f-b5bd-bb6c149deec9","Type":"ContainerDied","Data":"62e1e6eea5456513b2dc0b8a9a9475379de4f4b5184ee2e7edbf1042f414e85a"} Jul 06 14:32:09.840032 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.839694 2565 scope.go:117] "RemoveContainer" containerID="4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6" Jul 06 14:32:09.848148 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.848132 2565 scope.go:117] "RemoveContainer" containerID="4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6" Jul 06 14:32:09.848390 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:32:09.848372 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6\": container with ID starting with 4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6 not found: ID does not exist" containerID="4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6" Jul 06 14:32:09.848449 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.848404 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6"} err="failed to get container status \"4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6\": rpc error: code = NotFound desc = could not find container \"4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6\": container with ID starting with 4e7b2fc56b11f2d9ac41ccfe3f3b3c25da72ad9ccd9ceec12e8ca6a28f3a2dd6 not found: ID does not exist" Jul 06 14:32:09.867255 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.867226 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4"] Jul 06 14:32:09.872676 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:09.872653 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kxlw4"] Jul 06 14:32:10.932231 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:10.932197 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56ddda26-8bbf-4b0f-b5bd-bb6c149deec9" path="/var/lib/kubelet/pods/56ddda26-8bbf-4b0f-b5bd-bb6c149deec9/volumes" Jul 06 14:32:13.208920 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.208888 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz"] Jul 06 14:32:13.209354 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.209344 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56ddda26-8bbf-4b0f-b5bd-bb6c149deec9" containerName="manager" Jul 06 14:32:13.209420 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.209360 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="56ddda26-8bbf-4b0f-b5bd-bb6c149deec9" containerName="manager" Jul 06 14:32:13.209469 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.209428 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="56ddda26-8bbf-4b0f-b5bd-bb6c149deec9" containerName="manager" Jul 06 14:32:13.431860 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.431824 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz"] Jul 06 14:32:13.431860 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.431860 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.434684 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.434662 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qxnw9\"" Jul 06 14:32:13.506088 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506010 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.506088 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506046 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.506088 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506067 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.506302 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506116 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.506302 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506136 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcfe503a-8674-4fdb-9e90-0d13732db389-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.506302 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506210 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbf8q\" (UniqueName: \"kubernetes.io/projected/dcfe503a-8674-4fdb-9e90-0d13732db389-kube-api-access-bbf8q\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.506302 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506265 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.506302 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.506455 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.506307 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607026 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.606984 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607026 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607260 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607238 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607323 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607305 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607368 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607338 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607413 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607364 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607413 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607377 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607413 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607395 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcfe503a-8674-4fdb-9e90-0d13732db389-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607500 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbf8q\" (UniqueName: \"kubernetes.io/projected/dcfe503a-8674-4fdb-9e90-0d13732db389-kube-api-access-bbf8q\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607431 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607803 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607768 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.607905 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.607862 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.608125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.608107 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcfe503a-8674-4fdb-9e90-0d13732db389-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.610039 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.610013 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.610144 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.610089 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.617057 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.617034 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcfe503a-8674-4fdb-9e90-0d13732db389-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.617178 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.617038 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbf8q\" (UniqueName: \"kubernetes.io/projected/dcfe503a-8674-4fdb-9e90-0d13732db389-kube-api-access-bbf8q\") pod \"maas-default-gateway-openshift-default-56654dcf7-pnkwz\" (UID: \"dcfe503a-8674-4fdb-9e90-0d13732db389\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.741542 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.741505 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:13.898055 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:13.898020 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz"] Jul 06 14:32:13.902356 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:32:13.902324 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfe503a_8674_4fdb_9e90_0d13732db389.slice/crio-25799360456e503e12206a9ce5e1ea382195785455f2fd32ff0135b9786f9058 WatchSource:0}: Error finding container 25799360456e503e12206a9ce5e1ea382195785455f2fd32ff0135b9786f9058: Status 404 returned error can't find the container with id 25799360456e503e12206a9ce5e1ea382195785455f2fd32ff0135b9786f9058 Jul 06 14:32:14.868820 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:14.868759 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" event={"ID":"dcfe503a-8674-4fdb-9e90-0d13732db389","Type":"ContainerStarted","Data":"25799360456e503e12206a9ce5e1ea382195785455f2fd32ff0135b9786f9058"} Jul 06 14:32:17.307022 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:17.306981 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:32:17.307308 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:17.307062 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:32:17.307308 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:17.307089 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:32:17.884276 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:17.884242 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" event={"ID":"dcfe503a-8674-4fdb-9e90-0d13732db389","Type":"ContainerStarted","Data":"65e84486bf181cf42b8254ffbda152ee4a23d56e7585efc5677f381a7d0dcfd9"} Jul 06 14:32:17.912661 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:17.912612 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" podStartSLOduration=1.510492931 podStartE2EDuration="4.912593944s" totalImagesPullingTime="3.402101013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:32:17.909653771 +0000 UTC m=+473.452341332" watchObservedRunningTime="2026-07-06 14:32:17.912593944 +0000 UTC m=+473.455281483" Jul 06 14:32:18.742262 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:18.742225 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:18.747183 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:18.747159 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:18.889665 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:18.889627 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:18.890529 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:18.890513 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:32:28.635236 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.635198 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mwp5w"] Jul 06 14:32:28.639643 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.639625 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:28.642608 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.642585 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 14:32:28.649558 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.649525 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mwp5w"] Jul 06 14:32:28.725056 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.725018 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mwp5w"] Jul 06 14:32:28.742696 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.742668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz72l\" (UniqueName: \"kubernetes.io/projected/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-kube-api-access-qz72l\") pod \"limitador-limitador-64c94fd64f-mwp5w\" (UID: \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:28.742696 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.742700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-config-file\") pod \"limitador-limitador-64c94fd64f-mwp5w\" (UID: \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:28.843999 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.843953 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz72l\" (UniqueName: \"kubernetes.io/projected/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-kube-api-access-qz72l\") pod \"limitador-limitador-64c94fd64f-mwp5w\" (UID: \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:28.843999 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.843999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-config-file\") pod \"limitador-limitador-64c94fd64f-mwp5w\" (UID: \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:28.844602 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.844576 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-config-file\") pod \"limitador-limitador-64c94fd64f-mwp5w\" (UID: \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:28.852671 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.852640 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz72l\" (UniqueName: \"kubernetes.io/projected/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-kube-api-access-qz72l\") pod \"limitador-limitador-64c94fd64f-mwp5w\" (UID: \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:28.950396 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:28.950369 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:29.096485 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:29.096461 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mwp5w"] Jul 06 14:32:29.098901 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:32:29.098869 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbcb4e4_10e4_4416_99d5_d8f1ed417684.slice/crio-c10c12361bce605693fac3adc465c454c3746f1f14a9577914f8951d8e695fb2 WatchSource:0}: Error finding container c10c12361bce605693fac3adc465c454c3746f1f14a9577914f8951d8e695fb2: Status 404 returned error can't find the container with id c10c12361bce605693fac3adc465c454c3746f1f14a9577914f8951d8e695fb2 Jul 06 14:32:29.946575 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:29.946449 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" event={"ID":"bcbcb4e4-10e4-4416-99d5-d8f1ed417684","Type":"ContainerStarted","Data":"c10c12361bce605693fac3adc465c454c3746f1f14a9577914f8951d8e695fb2"} Jul 06 14:32:31.956465 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:31.956383 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" event={"ID":"bcbcb4e4-10e4-4416-99d5-d8f1ed417684","Type":"ContainerStarted","Data":"9571f840fd535b2ef3694338d60f11e248468ad89bb4828e98471d7a1d2440d5"} Jul 06 14:32:31.956849 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:31.956525 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:31.976062 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:31.976008 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" podStartSLOduration=1.369808978 podStartE2EDuration="3.975992645s" totalImagesPullingTime="2.606183667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:32:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:32:31.974390688 +0000 UTC m=+487.517078230" watchObservedRunningTime="2026-07-06 14:32:31.975992645 +0000 UTC m=+487.518680184" Jul 06 14:32:42.961104 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:42.961077 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:45.614377 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:45.614337 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mwp5w"] Jul 06 14:32:45.614942 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:45.614543 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" podUID="bcbcb4e4-10e4-4416-99d5-d8f1ed417684" containerName="limitador" containerID="cri-o://9571f840fd535b2ef3694338d60f11e248468ad89bb4828e98471d7a1d2440d5" gracePeriod=30 Jul 06 14:32:46.018827 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.018772 2565 generic.go:356] "Generic (PLEG): container finished" podID="bcbcb4e4-10e4-4416-99d5-d8f1ed417684" containerID="9571f840fd535b2ef3694338d60f11e248468ad89bb4828e98471d7a1d2440d5" exitCode=0 Jul 06 14:32:46.018990 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.018835 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" event={"ID":"bcbcb4e4-10e4-4416-99d5-d8f1ed417684","Type":"ContainerDied","Data":"9571f840fd535b2ef3694338d60f11e248468ad89bb4828e98471d7a1d2440d5"} Jul 06 14:32:46.565288 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.565261 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:46.576085 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.576062 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz72l\" (UniqueName: \"kubernetes.io/projected/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-kube-api-access-qz72l\") pod \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\" (UID: \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\") " Jul 06 14:32:46.576179 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.576120 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-config-file\") pod \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\" (UID: \"bcbcb4e4-10e4-4416-99d5-d8f1ed417684\") " Jul 06 14:32:46.576479 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.576447 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-config-file" (OuterVolumeSpecName: "config-file") pod "bcbcb4e4-10e4-4416-99d5-d8f1ed417684" (UID: "bcbcb4e4-10e4-4416-99d5-d8f1ed417684"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:32:46.578206 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.578177 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-kube-api-access-qz72l" (OuterVolumeSpecName: "kube-api-access-qz72l") pod "bcbcb4e4-10e4-4416-99d5-d8f1ed417684" (UID: "bcbcb4e4-10e4-4416-99d5-d8f1ed417684"). InnerVolumeSpecName "kube-api-access-qz72l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:32:46.677192 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.677156 2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-config-file\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:32:46.677192 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:46.677185 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz72l\" (UniqueName: \"kubernetes.io/projected/bcbcb4e4-10e4-4416-99d5-d8f1ed417684-kube-api-access-qz72l\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:32:47.024765 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:47.024722 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" event={"ID":"bcbcb4e4-10e4-4416-99d5-d8f1ed417684","Type":"ContainerDied","Data":"c10c12361bce605693fac3adc465c454c3746f1f14a9577914f8951d8e695fb2"} Jul 06 14:32:47.024941 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:47.024806 2565 scope.go:117] "RemoveContainer" containerID="9571f840fd535b2ef3694338d60f11e248468ad89bb4828e98471d7a1d2440d5" Jul 06 14:32:47.024941 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:47.024744 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mwp5w" Jul 06 14:32:47.048176 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:47.048143 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mwp5w"] Jul 06 14:32:47.054957 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:47.054928 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mwp5w"] Jul 06 14:32:48.931858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:48.931828 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcbcb4e4-10e4-4416-99d5-d8f1ed417684" path="/var/lib/kubelet/pods/bcbcb4e4-10e4-4416-99d5-d8f1ed417684/volumes" Jul 06 14:32:50.831833 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.831767 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-rrwm4"] Jul 06 14:32:50.832295 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.832149 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbcb4e4-10e4-4416-99d5-d8f1ed417684" containerName="limitador" Jul 06 14:32:50.832295 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.832162 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbcb4e4-10e4-4416-99d5-d8f1ed417684" containerName="limitador" Jul 06 14:32:50.832295 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.832232 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbcb4e4-10e4-4416-99d5-d8f1ed417684" containerName="limitador" Jul 06 14:32:50.838306 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.838281 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:50.841154 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.841138 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 14:32:50.841298 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.841255 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-v4sn7\"" Jul 06 14:32:50.852337 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.848501 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-rrwm4"] Jul 06 14:32:50.911987 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.911947 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8ad1a72c-32ba-4e87-a08a-2809561453a4-data\") pod \"postgres-69fb4594cf-rrwm4\" (UID: \"8ad1a72c-32ba-4e87-a08a-2809561453a4\") " pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:50.912172 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:50.912031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnkzs\" (UniqueName: \"kubernetes.io/projected/8ad1a72c-32ba-4e87-a08a-2809561453a4-kube-api-access-qnkzs\") pod \"postgres-69fb4594cf-rrwm4\" (UID: \"8ad1a72c-32ba-4e87-a08a-2809561453a4\") " pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:51.012856 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:51.012812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnkzs\" (UniqueName: \"kubernetes.io/projected/8ad1a72c-32ba-4e87-a08a-2809561453a4-kube-api-access-qnkzs\") pod \"postgres-69fb4594cf-rrwm4\" (UID: \"8ad1a72c-32ba-4e87-a08a-2809561453a4\") " pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:51.013059 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:51.012887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8ad1a72c-32ba-4e87-a08a-2809561453a4-data\") pod \"postgres-69fb4594cf-rrwm4\" (UID: \"8ad1a72c-32ba-4e87-a08a-2809561453a4\") " pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:51.013272 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:51.013250 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8ad1a72c-32ba-4e87-a08a-2809561453a4-data\") pod \"postgres-69fb4594cf-rrwm4\" (UID: \"8ad1a72c-32ba-4e87-a08a-2809561453a4\") " pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:51.023360 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:51.023332 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnkzs\" (UniqueName: \"kubernetes.io/projected/8ad1a72c-32ba-4e87-a08a-2809561453a4-kube-api-access-qnkzs\") pod \"postgres-69fb4594cf-rrwm4\" (UID: \"8ad1a72c-32ba-4e87-a08a-2809561453a4\") " pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:51.154156 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:51.154070 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:51.297623 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:51.297592 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-rrwm4"] Jul 06 14:32:51.300772 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:32:51.300743 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad1a72c_32ba_4e87_a08a_2809561453a4.slice/crio-59fab8c5d6f89327fb77ae06d636fc56c12c7aeef49d1e5e5949c39602bc1e57 WatchSource:0}: Error finding container 59fab8c5d6f89327fb77ae06d636fc56c12c7aeef49d1e5e5949c39602bc1e57: Status 404 returned error can't find the container with id 59fab8c5d6f89327fb77ae06d636fc56c12c7aeef49d1e5e5949c39602bc1e57 Jul 06 14:32:52.049797 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:52.049752 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-rrwm4" event={"ID":"8ad1a72c-32ba-4e87-a08a-2809561453a4","Type":"ContainerStarted","Data":"59fab8c5d6f89327fb77ae06d636fc56c12c7aeef49d1e5e5949c39602bc1e57"} Jul 06 14:32:57.077803 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:57.077745 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-rrwm4" event={"ID":"8ad1a72c-32ba-4e87-a08a-2809561453a4","Type":"ContainerStarted","Data":"ef10e75bc337242f4061e5d95cb6d5abad4bfa53396ffb42c1042cf7cd4b0c44"} Jul 06 14:32:57.078329 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:57.077932 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:32:57.097932 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:32:57.097871 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-rrwm4" podStartSLOduration=2.196861152 podStartE2EDuration="7.097848881s" totalImagesPullingTime="4.900987729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:32:57.095865836 +0000 UTC m=+512.638553373" watchObservedRunningTime="2026-07-06 14:32:57.097848881 +0000 UTC m=+512.640536420" Jul 06 14:33:03.112595 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:03.112562 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-rrwm4" Jul 06 14:33:10.166376 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.166331 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-746757b579-pjzkk"] Jul 06 14:33:10.174261 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.174233 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:10.177337 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.177310 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 14:33:10.177471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.177333 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 14:33:10.177471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.177354 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-znnsl\"" Jul 06 14:33:10.184307 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.184284 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746757b579-pjzkk"] Jul 06 14:33:10.290061 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.290025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-995wq\" (UniqueName: \"kubernetes.io/projected/5a607a04-639e-4200-9893-107ce5ce4e95-kube-api-access-995wq\") pod \"maas-controller-746757b579-pjzkk\" (UID: \"5a607a04-639e-4200-9893-107ce5ce4e95\") " pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:10.290231 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.290137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a607a04-639e-4200-9893-107ce5ce4e95-webhook-cert\") pod \"maas-controller-746757b579-pjzkk\" (UID: \"5a607a04-639e-4200-9893-107ce5ce4e95\") " pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:10.391269 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.391240 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a607a04-639e-4200-9893-107ce5ce4e95-webhook-cert\") pod \"maas-controller-746757b579-pjzkk\" (UID: \"5a607a04-639e-4200-9893-107ce5ce4e95\") " pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:10.391457 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.391292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-995wq\" (UniqueName: \"kubernetes.io/projected/5a607a04-639e-4200-9893-107ce5ce4e95-kube-api-access-995wq\") pod \"maas-controller-746757b579-pjzkk\" (UID: \"5a607a04-639e-4200-9893-107ce5ce4e95\") " pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:10.393745 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.393723 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a607a04-639e-4200-9893-107ce5ce4e95-webhook-cert\") pod \"maas-controller-746757b579-pjzkk\" (UID: \"5a607a04-639e-4200-9893-107ce5ce4e95\") " pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:10.402579 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.402549 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-995wq\" (UniqueName: \"kubernetes.io/projected/5a607a04-639e-4200-9893-107ce5ce4e95-kube-api-access-995wq\") pod \"maas-controller-746757b579-pjzkk\" (UID: \"5a607a04-639e-4200-9893-107ce5ce4e95\") " pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:10.486135 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.486091 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:10.623950 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:10.623923 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746757b579-pjzkk"] Jul 06 14:33:10.625862 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:33:10.625832 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a607a04_639e_4200_9893_107ce5ce4e95.slice/crio-e1412545eca134869a033b8fef618cfb638af435ecb0232c4e6b0ea069a36f2f WatchSource:0}: Error finding container e1412545eca134869a033b8fef618cfb638af435ecb0232c4e6b0ea069a36f2f: Status 404 returned error can't find the container with id e1412545eca134869a033b8fef618cfb638af435ecb0232c4e6b0ea069a36f2f Jul 06 14:33:11.138981 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:11.138938 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746757b579-pjzkk" event={"ID":"5a607a04-639e-4200-9893-107ce5ce4e95","Type":"ContainerStarted","Data":"e1412545eca134869a033b8fef618cfb638af435ecb0232c4e6b0ea069a36f2f"} Jul 06 14:33:13.149969 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:13.149938 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746757b579-pjzkk" event={"ID":"5a607a04-639e-4200-9893-107ce5ce4e95","Type":"ContainerStarted","Data":"7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93"} Jul 06 14:33:13.150353 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:13.150045 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:13.180816 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:13.180717 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-746757b579-pjzkk" podStartSLOduration=0.803789665 podStartE2EDuration="3.180698538s" totalImagesPullingTime="2.376908873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:13.17922294 +0000 UTC m=+528.721910481" watchObservedRunningTime="2026-07-06 14:33:13.180698538 +0000 UTC m=+528.723386076" Jul 06 14:33:24.162193 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:24.162112 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:26.109496 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.109462 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-746757b579-pjzkk"] Jul 06 14:33:26.109903 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.109680 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-746757b579-pjzkk" podUID="5a607a04-639e-4200-9893-107ce5ce4e95" containerName="manager" containerID="cri-o://7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93" gracePeriod=30 Jul 06 14:33:26.356530 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.356503 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:26.533745 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.533710 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-995wq\" (UniqueName: \"kubernetes.io/projected/5a607a04-639e-4200-9893-107ce5ce4e95-kube-api-access-995wq\") pod \"5a607a04-639e-4200-9893-107ce5ce4e95\" (UID: \"5a607a04-639e-4200-9893-107ce5ce4e95\") " Jul 06 14:33:26.533944 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.533773 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a607a04-639e-4200-9893-107ce5ce4e95-webhook-cert\") pod \"5a607a04-639e-4200-9893-107ce5ce4e95\" (UID: \"5a607a04-639e-4200-9893-107ce5ce4e95\") " Jul 06 14:33:26.536203 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.536176 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a607a04-639e-4200-9893-107ce5ce4e95-kube-api-access-995wq" (OuterVolumeSpecName: "kube-api-access-995wq") pod "5a607a04-639e-4200-9893-107ce5ce4e95" (UID: "5a607a04-639e-4200-9893-107ce5ce4e95"). InnerVolumeSpecName "kube-api-access-995wq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:33:26.536203 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.536190 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a607a04-639e-4200-9893-107ce5ce4e95-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5a607a04-639e-4200-9893-107ce5ce4e95" (UID: "5a607a04-639e-4200-9893-107ce5ce4e95"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:33:26.634777 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.634739 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-995wq\" (UniqueName: \"kubernetes.io/projected/5a607a04-639e-4200-9893-107ce5ce4e95-kube-api-access-995wq\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:33:26.634777 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:26.634771 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a607a04-639e-4200-9893-107ce5ce4e95-webhook-cert\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:33:27.212590 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.212505 2565 generic.go:356] "Generic (PLEG): container finished" podID="5a607a04-639e-4200-9893-107ce5ce4e95" containerID="7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93" exitCode=0 Jul 06 14:33:27.212590 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.212545 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746757b579-pjzkk" Jul 06 14:33:27.213218 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.212597 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746757b579-pjzkk" event={"ID":"5a607a04-639e-4200-9893-107ce5ce4e95","Type":"ContainerDied","Data":"7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93"} Jul 06 14:33:27.213218 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.212654 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746757b579-pjzkk" event={"ID":"5a607a04-639e-4200-9893-107ce5ce4e95","Type":"ContainerDied","Data":"e1412545eca134869a033b8fef618cfb638af435ecb0232c4e6b0ea069a36f2f"} Jul 06 14:33:27.213218 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.212688 2565 scope.go:117] "RemoveContainer" containerID="7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93" Jul 06 14:33:27.220746 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.220727 2565 scope.go:117] "RemoveContainer" containerID="7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93" Jul 06 14:33:27.221035 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:33:27.221015 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93\": container with ID starting with 7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93 not found: ID does not exist" containerID="7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93" Jul 06 14:33:27.221132 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.221041 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93"} err="failed to get container status \"7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93\": rpc error: code = NotFound desc = could not find container \"7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93\": container with ID starting with 7ab99ecd96ea7b82f9a4b11755fa17b7a63bc397140455e350d4474db7f4da93 not found: ID does not exist" Jul 06 14:33:27.232405 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.232380 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-746757b579-pjzkk"] Jul 06 14:33:27.239572 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:27.239549 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-746757b579-pjzkk"] Jul 06 14:33:28.932192 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:28.932160 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a607a04-639e-4200-9893-107ce5ce4e95" path="/var/lib/kubelet/pods/5a607a04-639e-4200-9893-107ce5ce4e95/volumes" Jul 06 14:33:46.776770 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.776736 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6f8d48684c-kjdd9"] Jul 06 14:33:46.777315 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.777270 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a607a04-639e-4200-9893-107ce5ce4e95" containerName="manager" Jul 06 14:33:46.777315 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.777289 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a607a04-639e-4200-9893-107ce5ce4e95" containerName="manager" Jul 06 14:33:46.777428 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.777391 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a607a04-639e-4200-9893-107ce5ce4e95" containerName="manager" Jul 06 14:33:46.795389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.795349 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6f8d48684c-kjdd9"] Jul 06 14:33:46.795534 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.795483 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:46.798607 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.798580 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 14:33:46.798804 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.798766 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-jt75v\"" Jul 06 14:33:46.897027 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.896996 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtftg\" (UniqueName: \"kubernetes.io/projected/88eafa99-3b3c-42bc-b23b-9dd6093ae909-kube-api-access-mtftg\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:46.897222 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.897041 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88eafa99-3b3c-42bc-b23b-9dd6093ae909-openshift-service-ca\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:46.897222 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.897083 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88eafa99-3b3c-42bc-b23b-9dd6093ae909-maas-api-tls\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:46.998168 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.998136 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88eafa99-3b3c-42bc-b23b-9dd6093ae909-maas-api-tls\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:46.998336 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.998195 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtftg\" (UniqueName: \"kubernetes.io/projected/88eafa99-3b3c-42bc-b23b-9dd6093ae909-kube-api-access-mtftg\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:46.998336 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.998225 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88eafa99-3b3c-42bc-b23b-9dd6093ae909-openshift-service-ca\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:46.998336 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:33:46.998317 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 06 14:33:46.998464 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:33:46.998413 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88eafa99-3b3c-42bc-b23b-9dd6093ae909-maas-api-tls podName:88eafa99-3b3c-42bc-b23b-9dd6093ae909 nodeName:}" failed. No retries permitted until 2026-07-06 14:33:47.498391337 +0000 UTC m=+563.041078869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/88eafa99-3b3c-42bc-b23b-9dd6093ae909-maas-api-tls") pod "maas-api-6f8d48684c-kjdd9" (UID: "88eafa99-3b3c-42bc-b23b-9dd6093ae909") : secret "maas-api-serving-cert" not found Jul 06 14:33:46.998986 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:46.998967 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88eafa99-3b3c-42bc-b23b-9dd6093ae909-openshift-service-ca\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:47.008199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.008174 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtftg\" (UniqueName: \"kubernetes.io/projected/88eafa99-3b3c-42bc-b23b-9dd6093ae909-kube-api-access-mtftg\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:47.332384 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.332346 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-pfcjl"] Jul 06 14:33:47.334588 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.334570 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:47.337736 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.337716 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 14:33:47.345974 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.345948 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-pfcjl"] Jul 06 14:33:47.375368 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.375334 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tbvng"] Jul 06 14:33:47.377491 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.377473 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:47.384724 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.384699 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zwb72\"" Jul 06 14:33:47.392814 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.392769 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tbvng"] Jul 06 14:33:47.402485 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.402461 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b9f8f57-3a1a-467a-a6d3-0fab2de54f82-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tbvng\" (UID: \"2b9f8f57-3a1a-467a-a6d3-0fab2de54f82\") " pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:47.402621 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.402504 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17e0d943-5abe-4fe1-a442-5b27b0776b65-plugins-config-volume\") pod \"payload-processing-58f86c7798-pfcjl\" (UID: \"17e0d943-5abe-4fe1-a442-5b27b0776b65\") " pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:47.402621 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.402528 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfccm\" (UniqueName: \"kubernetes.io/projected/17e0d943-5abe-4fe1-a442-5b27b0776b65-kube-api-access-pfccm\") pod \"payload-processing-58f86c7798-pfcjl\" (UID: \"17e0d943-5abe-4fe1-a442-5b27b0776b65\") " pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:47.402621 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.402597 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crb5v\" (UniqueName: \"kubernetes.io/projected/2b9f8f57-3a1a-467a-a6d3-0fab2de54f82-kube-api-access-crb5v\") pod \"payload-pre-processing-7964457696-tbvng\" (UID: \"2b9f8f57-3a1a-467a-a6d3-0fab2de54f82\") " pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:47.504163 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.504126 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17e0d943-5abe-4fe1-a442-5b27b0776b65-plugins-config-volume\") pod \"payload-processing-58f86c7798-pfcjl\" (UID: \"17e0d943-5abe-4fe1-a442-5b27b0776b65\") " pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:47.504344 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.504197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfccm\" (UniqueName: \"kubernetes.io/projected/17e0d943-5abe-4fe1-a442-5b27b0776b65-kube-api-access-pfccm\") pod \"payload-processing-58f86c7798-pfcjl\" (UID: \"17e0d943-5abe-4fe1-a442-5b27b0776b65\") " pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:47.504344 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.504228 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crb5v\" (UniqueName: \"kubernetes.io/projected/2b9f8f57-3a1a-467a-a6d3-0fab2de54f82-kube-api-access-crb5v\") pod \"payload-pre-processing-7964457696-tbvng\" (UID: \"2b9f8f57-3a1a-467a-a6d3-0fab2de54f82\") " pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:47.504344 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.504315 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88eafa99-3b3c-42bc-b23b-9dd6093ae909-maas-api-tls\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:47.504504 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.504376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b9f8f57-3a1a-467a-a6d3-0fab2de54f82-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tbvng\" (UID: \"2b9f8f57-3a1a-467a-a6d3-0fab2de54f82\") " pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:47.504802 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.504762 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17e0d943-5abe-4fe1-a442-5b27b0776b65-plugins-config-volume\") pod \"payload-processing-58f86c7798-pfcjl\" (UID: \"17e0d943-5abe-4fe1-a442-5b27b0776b65\") " pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:47.505152 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.505130 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b9f8f57-3a1a-467a-a6d3-0fab2de54f82-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tbvng\" (UID: \"2b9f8f57-3a1a-467a-a6d3-0fab2de54f82\") " pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:47.507259 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.507239 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88eafa99-3b3c-42bc-b23b-9dd6093ae909-maas-api-tls\") pod \"maas-api-6f8d48684c-kjdd9\" (UID: \"88eafa99-3b3c-42bc-b23b-9dd6093ae909\") " pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:47.513732 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.513605 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crb5v\" (UniqueName: \"kubernetes.io/projected/2b9f8f57-3a1a-467a-a6d3-0fab2de54f82-kube-api-access-crb5v\") pod \"payload-pre-processing-7964457696-tbvng\" (UID: \"2b9f8f57-3a1a-467a-a6d3-0fab2de54f82\") " pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:47.514027 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.514008 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfccm\" (UniqueName: \"kubernetes.io/projected/17e0d943-5abe-4fe1-a442-5b27b0776b65-kube-api-access-pfccm\") pod \"payload-processing-58f86c7798-pfcjl\" (UID: \"17e0d943-5abe-4fe1-a442-5b27b0776b65\") " pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:47.646860 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.646735 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:47.688486 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.688318 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:47.706712 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.706681 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:47.810589 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.809984 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-pfcjl"] Jul 06 14:33:47.863334 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.863273 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tbvng"] Jul 06 14:33:47.865622 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:33:47.865590 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9f8f57_3a1a_467a_a6d3_0fab2de54f82.slice/crio-2657b1e2323411b5109ca2b26668bc92d5b3518dcc44b237498308e1b9fc164c WatchSource:0}: Error finding container 2657b1e2323411b5109ca2b26668bc92d5b3518dcc44b237498308e1b9fc164c: Status 404 returned error can't find the container with id 2657b1e2323411b5109ca2b26668bc92d5b3518dcc44b237498308e1b9fc164c Jul 06 14:33:47.891226 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:47.890908 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6f8d48684c-kjdd9"] Jul 06 14:33:48.305746 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:48.305710 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" event={"ID":"2b9f8f57-3a1a-467a-a6d3-0fab2de54f82","Type":"ContainerStarted","Data":"2657b1e2323411b5109ca2b26668bc92d5b3518dcc44b237498308e1b9fc164c"} Jul 06 14:33:48.306875 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:48.306854 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6f8d48684c-kjdd9" event={"ID":"88eafa99-3b3c-42bc-b23b-9dd6093ae909","Type":"ContainerStarted","Data":"376208946d7621250d245549ed0c7d0de81464a83be8d2179d4a7ada2449ed99"} Jul 06 14:33:48.307982 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:48.307952 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" event={"ID":"17e0d943-5abe-4fe1-a442-5b27b0776b65","Type":"ContainerStarted","Data":"f75ec0185874f698e2f21ba271dbea453719af945fd21793f2a6a255d58d368e"} Jul 06 14:33:50.319830 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.319773 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" event={"ID":"17e0d943-5abe-4fe1-a442-5b27b0776b65","Type":"ContainerStarted","Data":"37b75dd6cc076945053d449a11510af7c466cf047c090aedf9c0b66b1ef04527"} Jul 06 14:33:50.320304 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.319895 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:33:50.321485 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.321460 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" event={"ID":"2b9f8f57-3a1a-467a-a6d3-0fab2de54f82","Type":"ContainerStarted","Data":"3c118fc40b72b0d052012181340cdd9aab756f28e5b918a21380a37bd5cfa91b"} Jul 06 14:33:50.321602 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.321526 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:33:50.322901 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.322881 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6f8d48684c-kjdd9" event={"ID":"88eafa99-3b3c-42bc-b23b-9dd6093ae909","Type":"ContainerStarted","Data":"a7981c6aa332484eaa3e5ad89c274d41fd599179766d73bdaa476f6091bed77e"} Jul 06 14:33:50.323023 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.323008 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:50.361925 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.360811 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" podStartSLOduration=1.8300954950000001 podStartE2EDuration="3.360770825s" totalImagesPullingTime="1.53067533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:50.359149631 +0000 UTC m=+565.901837171" watchObservedRunningTime="2026-07-06 14:33:50.360770825 +0000 UTC m=+565.903458365" Jul 06 14:33:50.404949 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.404836 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" podStartSLOduration=1.8984338520000001 podStartE2EDuration="3.404817885s" totalImagesPullingTime="1.506384033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:50.404400708 +0000 UTC m=+565.947088263" watchObservedRunningTime="2026-07-06 14:33:50.404817885 +0000 UTC m=+565.947505425" Jul 06 14:33:50.428585 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:50.428539 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6f8d48684c-kjdd9" podStartSLOduration=2.139118218 podStartE2EDuration="4.428522759s" totalImagesPullingTime="2.289404541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:50.426809694 +0000 UTC m=+565.969497225" watchObservedRunningTime="2026-07-06 14:33:50.428522759 +0000 UTC m=+565.971210297" Jul 06 14:33:55.347228 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:55.347192 2565 generic.go:356] "Generic (PLEG): container finished" podID="dcfe503a-8674-4fdb-9e90-0d13732db389" containerID="65e84486bf181cf42b8254ffbda152ee4a23d56e7585efc5677f381a7d0dcfd9" exitCode=0 Jul 06 14:33:55.347638 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:55.347268 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" event={"ID":"dcfe503a-8674-4fdb-9e90-0d13732db389","Type":"ContainerDied","Data":"65e84486bf181cf42b8254ffbda152ee4a23d56e7585efc5677f381a7d0dcfd9"} Jul 06 14:33:55.347638 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:55.347628 2565 scope.go:117] "RemoveContainer" containerID="65e84486bf181cf42b8254ffbda152ee4a23d56e7585efc5677f381a7d0dcfd9" Jul 06 14:33:55.348597 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:55.348570 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:33:55.348658 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:55.348644 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:33:55.348699 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:55.348684 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:33:56.333141 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:56.333109 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6f8d48684c-kjdd9" Jul 06 14:33:56.359273 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:56.359235 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" event={"ID":"dcfe503a-8674-4fdb-9e90-0d13732db389","Type":"ContainerStarted","Data":"7b41d0fea678559f0ed1a40d826ad55a4a4ae7e7868be8412354f7470c712cb0"} Jul 06 14:33:56.742091 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:56.742060 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:33:56.742253 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:56.742191 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-pnkwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 06 14:33:56.742253 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:56.742241 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" podUID="dcfe503a-8674-4fdb-9e90-0d13732db389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 06 14:33:57.742644 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:57.742611 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-pnkwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 06 14:33:57.743043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:57.742663 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" podUID="dcfe503a-8674-4fdb-9e90-0d13732db389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 06 14:33:58.742539 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:58.742503 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-pnkwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 06 14:33:58.742828 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:58.742557 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" podUID="dcfe503a-8674-4fdb-9e90-0d13732db389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 06 14:33:59.742177 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:59.742143 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-pnkwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 06 14:33:59.742350 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:33:59.742201 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" podUID="dcfe503a-8674-4fdb-9e90-0d13732db389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 06 14:34:00.746358 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:00.746323 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:34:00.746746 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:00.746591 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:34:00.747245 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:00.747228 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pnkwz" Jul 06 14:34:01.329986 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:01.329956 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-tbvng" Jul 06 14:34:01.330183 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:01.330037 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-pfcjl" Jul 06 14:34:30.197404 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.197373 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8"] Jul 06 14:34:30.200214 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.200196 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.202993 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.202966 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 14:34:30.204364 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.204347 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 14:34:30.204453 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.204351 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sg57q\"" Jul 06 14:34:30.204453 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.204408 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 14:34:30.210906 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.210885 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8"] Jul 06 14:34:30.277985 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.277952 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.277985 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.277983 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3640fc11-257a-453b-888d-83443aab1f53-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.278211 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.278025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.278211 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.278122 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjpt7\" (UniqueName: \"kubernetes.io/projected/3640fc11-257a-453b-888d-83443aab1f53-kube-api-access-jjpt7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.278211 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.278167 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.278211 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.278193 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.378941 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.378896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjpt7\" (UniqueName: \"kubernetes.io/projected/3640fc11-257a-453b-888d-83443aab1f53-kube-api-access-jjpt7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.379126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.378966 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.379126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.379000 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.379126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.379050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.379126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.379071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3640fc11-257a-453b-888d-83443aab1f53-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.379334 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.379128 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.379549 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.379521 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.382677 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.382643 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.382933 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.382911 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.383148 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.383127 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3640fc11-257a-453b-888d-83443aab1f53-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.385891 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.385869 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3640fc11-257a-453b-888d-83443aab1f53-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.389224 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.389199 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjpt7\" (UniqueName: \"kubernetes.io/projected/3640fc11-257a-453b-888d-83443aab1f53-kube-api-access-jjpt7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6s2c8\" (UID: \"3640fc11-257a-453b-888d-83443aab1f53\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.511564 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.511468 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:30.654049 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:30.654020 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8"] Jul 06 14:34:30.656216 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:34:30.656187 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3640fc11_257a_453b_888d_83443aab1f53.slice/crio-d0dbb3cbb4aa065d425f20ff7228450e2acd502a1dbcd9450875c1fd0a66d27d WatchSource:0}: Error finding container d0dbb3cbb4aa065d425f20ff7228450e2acd502a1dbcd9450875c1fd0a66d27d: Status 404 returned error can't find the container with id d0dbb3cbb4aa065d425f20ff7228450e2acd502a1dbcd9450875c1fd0a66d27d Jul 06 14:34:31.528339 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:31.528305 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" event={"ID":"3640fc11-257a-453b-888d-83443aab1f53","Type":"ContainerStarted","Data":"d0dbb3cbb4aa065d425f20ff7228450e2acd502a1dbcd9450875c1fd0a66d27d"} Jul 06 14:34:32.534578 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:32.534537 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" event={"ID":"3640fc11-257a-453b-888d-83443aab1f53","Type":"ContainerStarted","Data":"549cad2e69a165790b3ff0d82ded2250f8fd8b8235f8c17e930960a3b4c59bec"} Jul 06 14:34:32.554172 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:32.554126 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" podStartSLOduration=1.542927773 podStartE2EDuration="2.554110235s" totalImagesPullingTime="1.011182462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:34:32.552229758 +0000 UTC m=+608.094917296" watchObservedRunningTime="2026-07-06 14:34:32.554110235 +0000 UTC m=+608.096797773" Jul 06 14:34:40.511666 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.511620 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:40.512192 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.511718 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:40.524232 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.524203 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:40.582443 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.582417 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6s2c8" Jul 06 14:34:40.783279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.783203 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8"] Jul 06 14:34:40.785599 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.785583 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.788199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.788177 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 14:34:40.798238 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.798214 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8"] Jul 06 14:34:40.872804 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.872761 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b589bea-9a59-4b61-a999-ce459c8f24ad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.873005 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.872820 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.873005 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.872877 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.873005 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.872941 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.873005 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.872968 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.873005 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.873007 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxnpd\" (UniqueName: \"kubernetes.io/projected/8b589bea-9a59-4b61-a999-ce459c8f24ad-kube-api-access-gxnpd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974280 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974322 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974535 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974359 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974535 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974400 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxnpd\" (UniqueName: \"kubernetes.io/projected/8b589bea-9a59-4b61-a999-ce459c8f24ad-kube-api-access-gxnpd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974535 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b589bea-9a59-4b61-a999-ce459c8f24ad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974535 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974487 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974744 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974723 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974833 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974768 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.974885 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.974874 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.976825 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.976800 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b589bea-9a59-4b61-a999-ce459c8f24ad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.977061 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.977044 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b589bea-9a59-4b61-a999-ce459c8f24ad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:40.983928 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:40.983904 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxnpd\" (UniqueName: \"kubernetes.io/projected/8b589bea-9a59-4b61-a999-ce459c8f24ad-kube-api-access-gxnpd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8\" (UID: \"8b589bea-9a59-4b61-a999-ce459c8f24ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:41.097731 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:41.097646 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:41.233748 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:41.233723 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8"] Jul 06 14:34:41.236235 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:34:41.236202 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b589bea_9a59_4b61_a999_ce459c8f24ad.slice/crio-adee2d12214e21b8c416458f3e895031b828e8b84b51b670c5d2ad577d3b7784 WatchSource:0}: Error finding container adee2d12214e21b8c416458f3e895031b828e8b84b51b670c5d2ad577d3b7784: Status 404 returned error can't find the container with id adee2d12214e21b8c416458f3e895031b828e8b84b51b670c5d2ad577d3b7784 Jul 06 14:34:41.581898 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:41.581861 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" event={"ID":"8b589bea-9a59-4b61-a999-ce459c8f24ad","Type":"ContainerStarted","Data":"adee2d12214e21b8c416458f3e895031b828e8b84b51b670c5d2ad577d3b7784"} Jul 06 14:34:42.588021 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:42.587977 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" event={"ID":"8b589bea-9a59-4b61-a999-ce459c8f24ad","Type":"ContainerStarted","Data":"39ab9db8d263e2d04296df7a79a2d9ba30cd2887e00aeb78c0f5a672ae3f3255"} Jul 06 14:34:42.614660 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:42.614605 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" podStartSLOduration=2.3387457019999998 podStartE2EDuration="2.614589975s" totalImagesPullingTime="275.844273ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:34:42.613268913 +0000 UTC m=+618.155956451" watchObservedRunningTime="2026-07-06 14:34:42.614589975 +0000 UTC m=+618.157277512" Jul 06 14:34:44.993422 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:44.993385 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l"] Jul 06 14:34:44.995632 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:44.995616 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:44.998658 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:44.998634 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 14:34:45.006856 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.006832 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l"] Jul 06 14:34:45.112677 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.112647 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0cadaf1-851d-42ec-b619-a15d120fc826-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.112677 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.112679 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vsk6\" (UniqueName: \"kubernetes.io/projected/d0cadaf1-851d-42ec-b619-a15d120fc826-kube-api-access-9vsk6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.112897 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.112711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.112897 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.112750 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.112897 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.112767 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.112897 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.112808 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.213716 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.213674 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0cadaf1-851d-42ec-b619-a15d120fc826-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.213716 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.213717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vsk6\" (UniqueName: \"kubernetes.io/projected/d0cadaf1-851d-42ec-b619-a15d120fc826-kube-api-access-9vsk6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.213999 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.213748 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.213999 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.213813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.213999 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.213838 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.213999 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.213867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.214324 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.214286 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.214430 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.214297 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.214430 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.214361 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.216336 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.216313 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0cadaf1-851d-42ec-b619-a15d120fc826-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.216436 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.216321 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0cadaf1-851d-42ec-b619-a15d120fc826-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.227407 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.227374 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vsk6\" (UniqueName: \"kubernetes.io/projected/d0cadaf1-851d-42ec-b619-a15d120fc826-kube-api-access-9vsk6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lg87l\" (UID: \"d0cadaf1-851d-42ec-b619-a15d120fc826\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.306679 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.306561 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:45.443292 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.443267 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l"] Jul 06 14:34:45.445391 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:34:45.445362 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0cadaf1_851d_42ec_b619_a15d120fc826.slice/crio-ab90d06ae783d2388bcf7802f6aeececfd2713c6a86e770ee908f75f2e7d4b88 WatchSource:0}: Error finding container ab90d06ae783d2388bcf7802f6aeececfd2713c6a86e770ee908f75f2e7d4b88: Status 404 returned error can't find the container with id ab90d06ae783d2388bcf7802f6aeececfd2713c6a86e770ee908f75f2e7d4b88 Jul 06 14:34:45.446828 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.446812 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:34:45.603088 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:45.602985 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" event={"ID":"d0cadaf1-851d-42ec-b619-a15d120fc826","Type":"ContainerStarted","Data":"ab90d06ae783d2388bcf7802f6aeececfd2713c6a86e770ee908f75f2e7d4b88"} Jul 06 14:34:46.609182 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:46.609152 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" event={"ID":"d0cadaf1-851d-42ec-b619-a15d120fc826","Type":"ContainerStarted","Data":"49a42f006a5615fac1b7ff1d294fe72c5897e5bf0c18a513041b44d5a2bc9d70"} Jul 06 14:34:46.637145 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:46.637091 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" podStartSLOduration=2.390990838 podStartE2EDuration="2.637074415s" totalImagesPullingTime="246.083577ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:34:46.63475017 +0000 UTC m=+622.177437707" watchObservedRunningTime="2026-07-06 14:34:46.637074415 +0000 UTC m=+622.179761955" Jul 06 14:34:51.098080 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:51.098044 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:51.098555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:51.098116 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:51.110372 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:51.110348 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:51.649276 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:51.649247 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8" Jul 06 14:34:55.306833 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:55.306769 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:55.307285 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:55.306878 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:55.319581 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:55.319556 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:34:55.665082 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:34:55.665003 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lg87l" Jul 06 14:42:06.842682 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:06.842641 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn"] Jul 06 14:42:06.843240 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:06.842941 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" podUID="0dbbf7dd-d5c2-4516-ac38-74cabf19aa08" containerName="manager" containerID="cri-o://8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3" gracePeriod=10 Jul 06 14:42:09.608257 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.608235 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:42:09.634678 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.634570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-extensions-socket-volume\") pod \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\" (UID: \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\") " Jul 06 14:42:09.634678 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.634640 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bng4d\" (UniqueName: \"kubernetes.io/projected/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-kube-api-access-bng4d\") pod \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\" (UID: \"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08\") " Jul 06 14:42:09.635001 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.634949 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0dbbf7dd-d5c2-4516-ac38-74cabf19aa08" (UID: "0dbbf7dd-d5c2-4516-ac38-74cabf19aa08"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:09.637686 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.637639 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-kube-api-access-bng4d" (OuterVolumeSpecName: "kube-api-access-bng4d") pod "0dbbf7dd-d5c2-4516-ac38-74cabf19aa08" (UID: "0dbbf7dd-d5c2-4516-ac38-74cabf19aa08"). InnerVolumeSpecName "kube-api-access-bng4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:42:09.721416 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.721379 2565 generic.go:356] "Generic (PLEG): container finished" podID="0dbbf7dd-d5c2-4516-ac38-74cabf19aa08" containerID="8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3" exitCode=0 Jul 06 14:42:09.721588 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.721432 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" Jul 06 14:42:09.721588 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.721448 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" event={"ID":"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08","Type":"ContainerDied","Data":"8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3"} Jul 06 14:42:09.721588 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.721501 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn" event={"ID":"0dbbf7dd-d5c2-4516-ac38-74cabf19aa08","Type":"ContainerDied","Data":"8d8b4426f22242f428b954591c3d4085c319cb9bb87f8c57abe0c347b4687249"} Jul 06 14:42:09.721588 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.721531 2565 scope.go:117] "RemoveContainer" containerID="8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3" Jul 06 14:42:09.731416 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.731399 2565 scope.go:117] "RemoveContainer" containerID="8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3" Jul 06 14:42:09.731694 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:42:09.731675 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3\": container with ID starting with 8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3 not found: ID does not exist" containerID="8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3" Jul 06 14:42:09.731756 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.731703 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3"} err="failed to get container status \"8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3\": rpc error: code = NotFound desc = could not find container \"8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3\": container with ID starting with 8dbdc5ac234523e1e2e31378a48f2bac88ba80ea460c5b69c9db751f468520d3 not found: ID does not exist" Jul 06 14:42:09.736162 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.736138 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bng4d\" (UniqueName: \"kubernetes.io/projected/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-kube-api-access-bng4d\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:42:09.736162 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.736163 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08-extensions-socket-volume\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:42:09.746547 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.746523 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn"] Jul 06 14:42:09.751998 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:09.751973 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8xn"] Jul 06 14:42:10.931971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:42:10.931930 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dbbf7dd-d5c2-4516-ac38-74cabf19aa08" path="/var/lib/kubelet/pods/0dbbf7dd-d5c2-4516-ac38-74cabf19aa08/volumes" Jul 06 14:43:08.023507 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.023467 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd"] Jul 06 14:43:08.023947 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.023879 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbbf7dd-d5c2-4516-ac38-74cabf19aa08" containerName="manager" Jul 06 14:43:08.023947 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.023891 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbbf7dd-d5c2-4516-ac38-74cabf19aa08" containerName="manager" Jul 06 14:43:08.024028 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.023962 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbbf7dd-d5c2-4516-ac38-74cabf19aa08" containerName="manager" Jul 06 14:43:08.027043 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.027023 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:08.031060 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.031026 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dpjsc\"" Jul 06 14:43:08.089389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.089357 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd"] Jul 06 14:43:08.157250 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.157208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmh7r\" (UniqueName: \"kubernetes.io/projected/51d430df-1114-4aea-a6b9-74f405c0359c-kube-api-access-fmh7r\") pod \"kuadrant-operator-controller-manager-86878c769f-x5jsd\" (UID: \"51d430df-1114-4aea-a6b9-74f405c0359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:08.157423 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.157269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51d430df-1114-4aea-a6b9-74f405c0359c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x5jsd\" (UID: \"51d430df-1114-4aea-a6b9-74f405c0359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:08.258322 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.258281 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmh7r\" (UniqueName: \"kubernetes.io/projected/51d430df-1114-4aea-a6b9-74f405c0359c-kube-api-access-fmh7r\") pod \"kuadrant-operator-controller-manager-86878c769f-x5jsd\" (UID: \"51d430df-1114-4aea-a6b9-74f405c0359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:08.258484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.258332 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51d430df-1114-4aea-a6b9-74f405c0359c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x5jsd\" (UID: \"51d430df-1114-4aea-a6b9-74f405c0359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:08.258738 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.258717 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51d430df-1114-4aea-a6b9-74f405c0359c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x5jsd\" (UID: \"51d430df-1114-4aea-a6b9-74f405c0359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:08.271484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.271460 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmh7r\" (UniqueName: \"kubernetes.io/projected/51d430df-1114-4aea-a6b9-74f405c0359c-kube-api-access-fmh7r\") pod \"kuadrant-operator-controller-manager-86878c769f-x5jsd\" (UID: \"51d430df-1114-4aea-a6b9-74f405c0359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:08.337821 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.337706 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:08.484828 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.484804 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd"] Jul 06 14:43:08.486193 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:43:08.486149 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d430df_1114_4aea_a6b9_74f405c0359c.slice/crio-dfd49e0d7dececdb16a360deb1779498c53472e2276fd80fe9bb9de94baefab6 WatchSource:0}: Error finding container dfd49e0d7dececdb16a360deb1779498c53472e2276fd80fe9bb9de94baefab6: Status 404 returned error can't find the container with id dfd49e0d7dececdb16a360deb1779498c53472e2276fd80fe9bb9de94baefab6 Jul 06 14:43:08.999203 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.999170 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" event={"ID":"51d430df-1114-4aea-a6b9-74f405c0359c","Type":"ContainerStarted","Data":"67419911bc4bf5f8dd037afd1be77af10dcba958f2a6bce092e8990e1f3e7721"} Jul 06 14:43:08.999395 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.999223 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" event={"ID":"51d430df-1114-4aea-a6b9-74f405c0359c","Type":"ContainerStarted","Data":"dfd49e0d7dececdb16a360deb1779498c53472e2276fd80fe9bb9de94baefab6"} Jul 06 14:43:08.999395 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:08.999301 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:43:09.031348 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:09.031299 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" podStartSLOduration=1.031283662 podStartE2EDuration="1.031283662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:43:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:43:09.031033474 +0000 UTC m=+1124.573721014" watchObservedRunningTime="2026-07-06 14:43:09.031283662 +0000 UTC m=+1124.573971200" Jul 06 14:43:20.006223 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:43:20.006192 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x5jsd" Jul 06 14:49:05.753295 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.753256 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs"] Jul 06 14:49:05.757441 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.757422 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.760263 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.760238 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-36ce3e9f-openshift-default-dockercfg-bpmbj\"" Jul 06 14:49:05.776582 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.776552 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs"] Jul 06 14:49:05.935848 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.935815 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-socket\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.936023 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.935856 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-credential-socket\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.936023 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.935877 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-token\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.936023 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.935897 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-certs\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.936023 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.935942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-podinfo\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.936023 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.935968 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b11db7d-30dd-4785-91b0-d72f564a7c37-istiod-ca-cert\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.936023 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.935993 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwpq5\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-kube-api-access-pwpq5\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.936216 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.936027 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-envoy\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:05.936216 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:05.936054 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-data\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.036905 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.036813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b11db7d-30dd-4785-91b0-d72f564a7c37-istiod-ca-cert\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.036905 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.036849 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwpq5\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-kube-api-access-pwpq5\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.036905 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.036887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-envoy\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.036914 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-data\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.036962 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-socket\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.036982 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-credential-socket\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.037010 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-token\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.037038 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-certs\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.037076 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-podinfo\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037530 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.037353 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-data\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037530 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.037386 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-socket\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037530 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.037503 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-credential-socket\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037682 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.037602 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-certs\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.037682 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.037615 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b11db7d-30dd-4785-91b0-d72f564a7c37-istiod-ca-cert\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.039415 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.039386 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-envoy\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.039667 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.039647 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-podinfo\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.045856 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.045828 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-token\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.046435 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.046408 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwpq5\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-kube-api-access-pwpq5\") pod \"e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.069483 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.069447 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:06.211532 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.211498 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs"] Jul 06 14:49:06.214859 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:49:06.214831 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b11db7d_30dd_4785_91b0_d72f564a7c37.slice/crio-3c7379fcc9f62656711dfa00f11bc99207fc349fd07f8f575fe54c7e5a78facc WatchSource:0}: Error finding container 3c7379fcc9f62656711dfa00f11bc99207fc349fd07f8f575fe54c7e5a78facc: Status 404 returned error can't find the container with id 3c7379fcc9f62656711dfa00f11bc99207fc349fd07f8f575fe54c7e5a78facc Jul 06 14:49:06.216985 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.216954 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:49:06.217058 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.217022 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:49:06.217093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.217055 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:49:06.688692 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.688654 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" event={"ID":"1b11db7d-30dd-4785-91b0-d72f564a7c37","Type":"ContainerStarted","Data":"330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321"} Jul 06 14:49:06.688876 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.688707 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" event={"ID":"1b11db7d-30dd-4785-91b0-d72f564a7c37","Type":"ContainerStarted","Data":"3c7379fcc9f62656711dfa00f11bc99207fc349fd07f8f575fe54c7e5a78facc"} Jul 06 14:49:06.723749 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:06.723692 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" podStartSLOduration=1.723670297 podStartE2EDuration="1.723670297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:06.723295686 +0000 UTC m=+1482.265983235" watchObservedRunningTime="2026-07-06 14:49:06.723670297 +0000 UTC m=+1482.266357837" Jul 06 14:49:07.070485 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.070394 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:07.075429 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.075404 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:07.290311 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.290269 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml"] Jul 06 14:49:07.294578 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.294552 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.297397 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.297375 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-36ce3e9f-serving-cert\"" Jul 06 14:49:07.305417 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.305393 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml"] Jul 06 14:49:07.450075 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.450039 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f840a5e-e90f-4011-b1e3-dc5701f88adc-openshift-service-ca\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.450254 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.450091 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f840a5e-e90f-4011-b1e3-dc5701f88adc-maas-api-tls\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.450254 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.450148 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9pzc\" (UniqueName: \"kubernetes.io/projected/7f840a5e-e90f-4011-b1e3-dc5701f88adc-kube-api-access-k9pzc\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.551281 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.551245 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9pzc\" (UniqueName: \"kubernetes.io/projected/7f840a5e-e90f-4011-b1e3-dc5701f88adc-kube-api-access-k9pzc\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.551464 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.551364 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f840a5e-e90f-4011-b1e3-dc5701f88adc-openshift-service-ca\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.551464 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.551416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f840a5e-e90f-4011-b1e3-dc5701f88adc-maas-api-tls\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.552081 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.552060 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f840a5e-e90f-4011-b1e3-dc5701f88adc-openshift-service-ca\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.554062 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.554039 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f840a5e-e90f-4011-b1e3-dc5701f88adc-maas-api-tls\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.560603 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.560576 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9pzc\" (UniqueName: \"kubernetes.io/projected/7f840a5e-e90f-4011-b1e3-dc5701f88adc-kube-api-access-k9pzc\") pod \"maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.606814 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.606743 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:07.694436 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.694288 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:07.695680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.695654 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:07.764298 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.764265 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml"] Jul 06 14:49:07.767369 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:49:07.766516 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f840a5e_e90f_4011_b1e3_dc5701f88adc.slice/crio-3687046fc23c14dddc042ac4b609cdaeef36367bbd8f030773b1a6a5a1820c8e WatchSource:0}: Error finding container 3687046fc23c14dddc042ac4b609cdaeef36367bbd8f030773b1a6a5a1820c8e: Status 404 returned error can't find the container with id 3687046fc23c14dddc042ac4b609cdaeef36367bbd8f030773b1a6a5a1820c8e Jul 06 14:49:07.768266 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:07.768248 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:49:08.700548 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:08.700506 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" event={"ID":"7f840a5e-e90f-4011-b1e3-dc5701f88adc","Type":"ContainerStarted","Data":"d032aeaf1bc5937d90a14cd022d803b508cc14deba19fb8dfcd21115fb9bcce5"} Jul 06 14:49:08.701020 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:08.700571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" event={"ID":"7f840a5e-e90f-4011-b1e3-dc5701f88adc","Type":"ContainerStarted","Data":"3687046fc23c14dddc042ac4b609cdaeef36367bbd8f030773b1a6a5a1820c8e"} Jul 06 14:49:08.701020 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:08.700885 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:08.719536 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:08.719489 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" podStartSLOduration=1.072643486 podStartE2EDuration="1.719472276s" totalImagesPullingTime="646.82879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:08.71782465 +0000 UTC m=+1484.260512189" watchObservedRunningTime="2026-07-06 14:49:08.719472276 +0000 UTC m=+1484.262159814" Jul 06 14:49:12.314373 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.314331 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs"] Jul 06 14:49:12.314930 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.314512 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" podUID="1b11db7d-30dd-4785-91b0-d72f564a7c37" containerName="istio-proxy" containerID="cri-o://330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321" gracePeriod=30 Jul 06 14:49:12.521844 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.521806 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml"] Jul 06 14:49:12.522082 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.522050 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" podUID="7f840a5e-e90f-4011-b1e3-dc5701f88adc" containerName="maas-api" containerID="cri-o://d032aeaf1bc5937d90a14cd022d803b508cc14deba19fb8dfcd21115fb9bcce5" gracePeriod=30 Jul 06 14:49:12.723484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.723445 2565 generic.go:356] "Generic (PLEG): container finished" podID="7f840a5e-e90f-4011-b1e3-dc5701f88adc" containerID="d032aeaf1bc5937d90a14cd022d803b508cc14deba19fb8dfcd21115fb9bcce5" exitCode=0 Jul 06 14:49:12.723631 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.723481 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" event={"ID":"7f840a5e-e90f-4011-b1e3-dc5701f88adc","Type":"ContainerDied","Data":"d032aeaf1bc5937d90a14cd022d803b508cc14deba19fb8dfcd21115fb9bcce5"} Jul 06 14:49:12.834086 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.834064 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:12.901631 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.901538 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9pzc\" (UniqueName: \"kubernetes.io/projected/7f840a5e-e90f-4011-b1e3-dc5701f88adc-kube-api-access-k9pzc\") pod \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " Jul 06 14:49:12.901631 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.901606 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f840a5e-e90f-4011-b1e3-dc5701f88adc-maas-api-tls\") pod \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " Jul 06 14:49:12.901890 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.901667 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f840a5e-e90f-4011-b1e3-dc5701f88adc-openshift-service-ca\") pod \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\" (UID: \"7f840a5e-e90f-4011-b1e3-dc5701f88adc\") " Jul 06 14:49:12.902092 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.902063 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f840a5e-e90f-4011-b1e3-dc5701f88adc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f840a5e-e90f-4011-b1e3-dc5701f88adc" (UID: "7f840a5e-e90f-4011-b1e3-dc5701f88adc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:12.903773 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.903746 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f840a5e-e90f-4011-b1e3-dc5701f88adc-kube-api-access-k9pzc" (OuterVolumeSpecName: "kube-api-access-k9pzc") pod "7f840a5e-e90f-4011-b1e3-dc5701f88adc" (UID: "7f840a5e-e90f-4011-b1e3-dc5701f88adc"). InnerVolumeSpecName "kube-api-access-k9pzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:12.903906 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:12.903874 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f840a5e-e90f-4011-b1e3-dc5701f88adc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f840a5e-e90f-4011-b1e3-dc5701f88adc" (UID: "7f840a5e-e90f-4011-b1e3-dc5701f88adc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:49:13.002291 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:13.002256 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f840a5e-e90f-4011-b1e3-dc5701f88adc-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:13.002291 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:13.002286 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9pzc\" (UniqueName: \"kubernetes.io/projected/7f840a5e-e90f-4011-b1e3-dc5701f88adc-kube-api-access-k9pzc\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:13.002291 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:13.002296 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f840a5e-e90f-4011-b1e3-dc5701f88adc-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:13.729354 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:13.729322 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" event={"ID":"7f840a5e-e90f-4011-b1e3-dc5701f88adc","Type":"ContainerDied","Data":"3687046fc23c14dddc042ac4b609cdaeef36367bbd8f030773b1a6a5a1820c8e"} Jul 06 14:49:13.729743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:13.729378 2565 scope.go:117] "RemoveContainer" containerID="d032aeaf1bc5937d90a14cd022d803b508cc14deba19fb8dfcd21115fb9bcce5" Jul 06 14:49:13.729743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:13.729333 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml" Jul 06 14:49:13.752453 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:13.752421 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml"] Jul 06 14:49:13.764362 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:13.764323 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-36ce3e9f-5b9bf895c6-nt8ml"] Jul 06 14:49:14.932945 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:14.932868 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f840a5e-e90f-4011-b1e3-dc5701f88adc" path="/var/lib/kubelet/pods/7f840a5e-e90f-4011-b1e3-dc5701f88adc/volumes" Jul 06 14:49:17.575082 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.575058 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:17.646655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646575 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-credential-socket\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646622 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-certs\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646646 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-socket\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646946 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646675 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-token\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646946 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646693 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-data\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646946 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646715 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-podinfo\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646946 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646770 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b11db7d-30dd-4785-91b0-d72f564a7c37-istiod-ca-cert\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646946 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646827 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwpq5\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-kube-api-access-pwpq5\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646946 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646886 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-envoy\") pod \"1b11db7d-30dd-4785-91b0-d72f564a7c37\" (UID: \"1b11db7d-30dd-4785-91b0-d72f564a7c37\") " Jul 06 14:49:17.646946 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646929 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:17.647296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.646948 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:17.647296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.647093 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:17.647296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.647238 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-credential-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.647296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.647260 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-certs\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.647296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.647274 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-workload-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.647296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.647265 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b11db7d-30dd-4785-91b0-d72f564a7c37-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:17.647691 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.647658 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:17.649167 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.649139 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:17.649269 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.649140 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-kube-api-access-pwpq5" (OuterVolumeSpecName: "kube-api-access-pwpq5") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "kube-api-access-pwpq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:17.649269 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.649188 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:17.649628 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.649608 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b11db7d-30dd-4785-91b0-d72f564a7c37" (UID: "1b11db7d-30dd-4785-91b0-d72f564a7c37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:49:17.748501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.748466 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-token\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.748501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.748502 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-data\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.748501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.748518 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-podinfo\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.748735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.748532 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b11db7d-30dd-4785-91b0-d72f564a7c37-istiod-ca-cert\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.748735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.748548 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwpq5\" (UniqueName: \"kubernetes.io/projected/1b11db7d-30dd-4785-91b0-d72f564a7c37-kube-api-access-pwpq5\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.748735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.748561 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b11db7d-30dd-4785-91b0-d72f564a7c37-istio-envoy\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:17.751951 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.751918 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b11db7d-30dd-4785-91b0-d72f564a7c37" containerID="330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321" exitCode=0 Jul 06 14:49:17.752073 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.751973 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" event={"ID":"1b11db7d-30dd-4785-91b0-d72f564a7c37","Type":"ContainerDied","Data":"330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321"} Jul 06 14:49:17.752073 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.751978 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" Jul 06 14:49:17.752073 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.752017 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs" event={"ID":"1b11db7d-30dd-4785-91b0-d72f564a7c37","Type":"ContainerDied","Data":"3c7379fcc9f62656711dfa00f11bc99207fc349fd07f8f575fe54c7e5a78facc"} Jul 06 14:49:17.752073 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.752038 2565 scope.go:117] "RemoveContainer" containerID="330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321" Jul 06 14:49:17.761814 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.761774 2565 scope.go:117] "RemoveContainer" containerID="330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321" Jul 06 14:49:17.762129 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:49:17.762109 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321\": container with ID starting with 330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321 not found: ID does not exist" containerID="330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321" Jul 06 14:49:17.762196 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.762137 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321"} err="failed to get container status \"330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321\": rpc error: code = NotFound desc = could not find container \"330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321\": container with ID starting with 330d8a99edc50eb348643a1c2ae61270ea6ee0db79f83574c93473eef45fb321 not found: ID does not exist" Jul 06 14:49:17.777015 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.776983 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs"] Jul 06 14:49:17.782865 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:17.782837 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-36ce3e9f-openshift-default-7d7dbd8587-nbfrs"] Jul 06 14:49:18.932859 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:18.932826 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b11db7d-30dd-4785-91b0-d72f564a7c37" path="/var/lib/kubelet/pods/1b11db7d-30dd-4785-91b0-d72f564a7c37/volumes" Jul 06 14:49:20.224728 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.224691 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg"] Jul 06 14:49:20.225297 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.225274 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f840a5e-e90f-4011-b1e3-dc5701f88adc" containerName="maas-api" Jul 06 14:49:20.225341 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.225302 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f840a5e-e90f-4011-b1e3-dc5701f88adc" containerName="maas-api" Jul 06 14:49:20.225341 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.225322 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b11db7d-30dd-4785-91b0-d72f564a7c37" containerName="istio-proxy" Jul 06 14:49:20.225341 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.225331 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b11db7d-30dd-4785-91b0-d72f564a7c37" containerName="istio-proxy" Jul 06 14:49:20.225457 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.225444 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f840a5e-e90f-4011-b1e3-dc5701f88adc" containerName="maas-api" Jul 06 14:49:20.225495 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.225468 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b11db7d-30dd-4785-91b0-d72f564a7c37" containerName="istio-proxy" Jul 06 14:49:20.230895 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.230869 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.234022 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.234002 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-df5e6c40-serving-cert\"" Jul 06 14:49:20.245540 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.245512 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg"] Jul 06 14:49:20.272096 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.272061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b698bc3-9a80-4cda-af0c-552692f0b975-openshift-service-ca\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.272273 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.272200 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.272273 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.272240 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhtsn\" (UniqueName: \"kubernetes.io/projected/1b698bc3-9a80-4cda-af0c-552692f0b975-kube-api-access-zhtsn\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.372897 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.372860 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhtsn\" (UniqueName: \"kubernetes.io/projected/1b698bc3-9a80-4cda-af0c-552692f0b975-kube-api-access-zhtsn\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.373096 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.372938 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b698bc3-9a80-4cda-af0c-552692f0b975-openshift-service-ca\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.373096 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.372992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.373096 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:49:20.373079 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-df5e6c40-serving-cert: secret "maas-api-e2e-ait-df5e6c40-serving-cert" not found Jul 06 14:49:20.373267 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:49:20.373133 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls podName:1b698bc3-9a80-4cda-af0c-552692f0b975 nodeName:}" failed. No retries permitted until 2026-07-06 14:49:20.873117057 +0000 UTC m=+1496.415804572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls") pod "maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" (UID: "1b698bc3-9a80-4cda-af0c-552692f0b975") : secret "maas-api-e2e-ait-df5e6c40-serving-cert" not found Jul 06 14:49:20.373687 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.373661 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b698bc3-9a80-4cda-af0c-552692f0b975-openshift-service-ca\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.394272 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.394242 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhtsn\" (UniqueName: \"kubernetes.io/projected/1b698bc3-9a80-4cda-af0c-552692f0b975-kube-api-access-zhtsn\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.878500 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.878470 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:20.881046 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:20.881013 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls\") pod \"maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:21.142894 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:21.142773 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:21.292056 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:21.292028 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg"] Jul 06 14:49:21.294303 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:49:21.294278 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b698bc3_9a80_4cda_af0c_552692f0b975.slice/crio-ecd166c7dace47705b5d632335c1a8e831a53ea729ebc904cdb47e3fea8e512d WatchSource:0}: Error finding container ecd166c7dace47705b5d632335c1a8e831a53ea729ebc904cdb47e3fea8e512d: Status 404 returned error can't find the container with id ecd166c7dace47705b5d632335c1a8e831a53ea729ebc904cdb47e3fea8e512d Jul 06 14:49:21.775390 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:21.775356 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" event={"ID":"1b698bc3-9a80-4cda-af0c-552692f0b975","Type":"ContainerStarted","Data":"ecd166c7dace47705b5d632335c1a8e831a53ea729ebc904cdb47e3fea8e512d"} Jul 06 14:49:22.782164 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:22.782127 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" event={"ID":"1b698bc3-9a80-4cda-af0c-552692f0b975","Type":"ContainerStarted","Data":"8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd"} Jul 06 14:49:22.782574 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:22.782224 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:22.802718 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:22.802662 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" podStartSLOduration=2.376808159 podStartE2EDuration="2.802646583s" totalImagesPullingTime="425.838424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:22.801512237 +0000 UTC m=+1498.344199776" watchObservedRunningTime="2026-07-06 14:49:22.802646583 +0000 UTC m=+1498.345334120" Jul 06 14:49:25.799894 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:25.799853 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg"] Jul 06 14:49:25.800387 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:25.800067 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" podUID="1b698bc3-9a80-4cda-af0c-552692f0b975" containerName="maas-api" containerID="cri-o://8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd" gracePeriod=30 Jul 06 14:49:26.055991 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.055916 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:26.128387 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.128355 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhtsn\" (UniqueName: \"kubernetes.io/projected/1b698bc3-9a80-4cda-af0c-552692f0b975-kube-api-access-zhtsn\") pod \"1b698bc3-9a80-4cda-af0c-552692f0b975\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " Jul 06 14:49:26.128541 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.128483 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b698bc3-9a80-4cda-af0c-552692f0b975-openshift-service-ca\") pod \"1b698bc3-9a80-4cda-af0c-552692f0b975\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " Jul 06 14:49:26.128604 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.128552 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls\") pod \"1b698bc3-9a80-4cda-af0c-552692f0b975\" (UID: \"1b698bc3-9a80-4cda-af0c-552692f0b975\") " Jul 06 14:49:26.128861 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.128828 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b698bc3-9a80-4cda-af0c-552692f0b975-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b698bc3-9a80-4cda-af0c-552692f0b975" (UID: "1b698bc3-9a80-4cda-af0c-552692f0b975"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:26.130668 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.130638 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b698bc3-9a80-4cda-af0c-552692f0b975-kube-api-access-zhtsn" (OuterVolumeSpecName: "kube-api-access-zhtsn") pod "1b698bc3-9a80-4cda-af0c-552692f0b975" (UID: "1b698bc3-9a80-4cda-af0c-552692f0b975"). InnerVolumeSpecName "kube-api-access-zhtsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:26.130758 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.130681 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b698bc3-9a80-4cda-af0c-552692f0b975" (UID: "1b698bc3-9a80-4cda-af0c-552692f0b975"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:49:26.230064 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.230022 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b698bc3-9a80-4cda-af0c-552692f0b975-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:26.230064 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.230056 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b698bc3-9a80-4cda-af0c-552692f0b975-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:26.230064 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.230066 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhtsn\" (UniqueName: \"kubernetes.io/projected/1b698bc3-9a80-4cda-af0c-552692f0b975-kube-api-access-zhtsn\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:26.804110 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.804075 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b698bc3-9a80-4cda-af0c-552692f0b975" containerID="8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd" exitCode=0 Jul 06 14:49:26.804594 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.804122 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" Jul 06 14:49:26.804594 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.804120 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" event={"ID":"1b698bc3-9a80-4cda-af0c-552692f0b975","Type":"ContainerDied","Data":"8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd"} Jul 06 14:49:26.804594 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.804188 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg" event={"ID":"1b698bc3-9a80-4cda-af0c-552692f0b975","Type":"ContainerDied","Data":"ecd166c7dace47705b5d632335c1a8e831a53ea729ebc904cdb47e3fea8e512d"} Jul 06 14:49:26.804594 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.804219 2565 scope.go:117] "RemoveContainer" containerID="8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd" Jul 06 14:49:26.814337 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.814318 2565 scope.go:117] "RemoveContainer" containerID="8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd" Jul 06 14:49:26.814622 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:49:26.814603 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd\": container with ID starting with 8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd not found: ID does not exist" containerID="8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd" Jul 06 14:49:26.814681 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.814633 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd"} err="failed to get container status \"8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd\": rpc error: code = NotFound desc = could not find container \"8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd\": container with ID starting with 8157bc7cf00ad0037acce0cd02cc5614f482a33fd131a9882bfe2d731afc80bd not found: ID does not exist" Jul 06 14:49:26.827728 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.827710 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg"] Jul 06 14:49:26.831932 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.831912 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-df5e6c40-84d546bdff-9hmqg"] Jul 06 14:49:26.932632 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:26.932602 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b698bc3-9a80-4cda-af0c-552692f0b975" path="/var/lib/kubelet/pods/1b698bc3-9a80-4cda-af0c-552692f0b975/volumes" Jul 06 14:49:38.784176 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.784142 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz"] Jul 06 14:49:38.784687 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.784513 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b698bc3-9a80-4cda-af0c-552692f0b975" containerName="maas-api" Jul 06 14:49:38.784687 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.784527 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b698bc3-9a80-4cda-af0c-552692f0b975" containerName="maas-api" Jul 06 14:49:38.784687 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.784592 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b698bc3-9a80-4cda-af0c-552692f0b975" containerName="maas-api" Jul 06 14:49:38.789187 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.789165 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.792538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.792517 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9c941945-openshift-default-dockercfg-zgwdz\"" Jul 06 14:49:38.805458 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.805432 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz"] Jul 06 14:49:38.955629 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955596 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6fzm\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-kube-api-access-z6fzm\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.955840 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955637 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-token\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.955840 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955654 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istiod-ca-cert\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.955840 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955716 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-envoy\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.955840 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955771 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-certs\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.955840 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955835 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-data\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.956028 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-credential-socket\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.956028 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-podinfo\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:38.956028 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:38.955924 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-socket\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.056644 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.056550 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6fzm\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-kube-api-access-z6fzm\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.056644 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.056606 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-token\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.056644 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.056634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istiod-ca-cert\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.056953 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.056659 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-envoy\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.056953 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.056696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-certs\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057179 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057154 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-data\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057289 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057268 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-credential-socket\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057347 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057304 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-podinfo\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057397 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057341 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-certs\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057397 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057373 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-socket\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057500 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057407 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-data\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057500 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057478 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istiod-ca-cert\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057720 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057699 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-socket\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.057969 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.057947 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-credential-socket\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.059219 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.059199 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-envoy\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.059651 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.059631 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-podinfo\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.066208 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.066181 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-token\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.066287 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.066256 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6fzm\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-kube-api-access-z6fzm\") pod \"e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.101662 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.101626 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:39.252619 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.252585 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz"] Jul 06 14:49:39.255720 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:49:39.255686 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda61a2370_06f5_438c_ae6f_4d6d55fabbfa.slice/crio-5d1954604e1e9ed7628e3caea5f2c0e95622f7fe1cef78e4ff27c156ffddd568 WatchSource:0}: Error finding container 5d1954604e1e9ed7628e3caea5f2c0e95622f7fe1cef78e4ff27c156ffddd568: Status 404 returned error can't find the container with id 5d1954604e1e9ed7628e3caea5f2c0e95622f7fe1cef78e4ff27c156ffddd568 Jul 06 14:49:39.258403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.258363 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:49:39.258497 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.258455 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:49:39.258553 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.258498 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:49:39.871616 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.871578 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" event={"ID":"a61a2370-06f5-438c-ae6f-4d6d55fabbfa","Type":"ContainerStarted","Data":"68e0d8ab56c6694920c96b3c27c4a9076707b9f61fb5a1ca3f24ea07773912b9"} Jul 06 14:49:39.872210 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.871637 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" event={"ID":"a61a2370-06f5-438c-ae6f-4d6d55fabbfa","Type":"ContainerStarted","Data":"5d1954604e1e9ed7628e3caea5f2c0e95622f7fe1cef78e4ff27c156ffddd568"} Jul 06 14:49:39.906508 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:39.906456 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" podStartSLOduration=1.9064397400000002 podStartE2EDuration="1.90643974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:39.906091015 +0000 UTC m=+1515.448778553" watchObservedRunningTime="2026-07-06 14:49:39.90643974 +0000 UTC m=+1515.449127277" Jul 06 14:49:40.102610 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.102572 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:40.107424 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.107400 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:40.215649 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.215616 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv"] Jul 06 14:49:40.219162 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.219128 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.227172 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.227146 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-9c941945-serving-cert\"" Jul 06 14:49:40.248900 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.248709 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv"] Jul 06 14:49:40.371090 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.371047 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmzkw\" (UniqueName: \"kubernetes.io/projected/7136b9c3-8b01-4279-9dbc-b37a92db6317-kube-api-access-jmzkw\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.371289 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.371169 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7136b9c3-8b01-4279-9dbc-b37a92db6317-openshift-service-ca\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.371289 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.371198 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7136b9c3-8b01-4279-9dbc-b37a92db6317-maas-api-tls\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.472055 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.471956 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmzkw\" (UniqueName: \"kubernetes.io/projected/7136b9c3-8b01-4279-9dbc-b37a92db6317-kube-api-access-jmzkw\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.472055 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.472050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7136b9c3-8b01-4279-9dbc-b37a92db6317-openshift-service-ca\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.472297 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.472076 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7136b9c3-8b01-4279-9dbc-b37a92db6317-maas-api-tls\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.472717 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.472682 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7136b9c3-8b01-4279-9dbc-b37a92db6317-openshift-service-ca\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.474592 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.474569 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7136b9c3-8b01-4279-9dbc-b37a92db6317-maas-api-tls\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.492531 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.492502 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmzkw\" (UniqueName: \"kubernetes.io/projected/7136b9c3-8b01-4279-9dbc-b37a92db6317-kube-api-access-jmzkw\") pod \"maas-api-e2e-derive-9c941945-d748787bf-bg6zv\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.531707 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.531670 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:40.699329 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.699294 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv"] Jul 06 14:49:40.703115 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:49:40.703087 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7136b9c3_8b01_4279_9dbc_b37a92db6317.slice/crio-9a734a599ff72fe319d9bb96204732f8475f1fc97cf52b83befb5d172816c8ea WatchSource:0}: Error finding container 9a734a599ff72fe319d9bb96204732f8475f1fc97cf52b83befb5d172816c8ea: Status 404 returned error can't find the container with id 9a734a599ff72fe319d9bb96204732f8475f1fc97cf52b83befb5d172816c8ea Jul 06 14:49:40.877691 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.877598 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" event={"ID":"7136b9c3-8b01-4279-9dbc-b37a92db6317","Type":"ContainerStarted","Data":"9a734a599ff72fe319d9bb96204732f8475f1fc97cf52b83befb5d172816c8ea"} Jul 06 14:49:40.878093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.877813 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:40.878726 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:40.878711 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:41.884240 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:41.884197 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" event={"ID":"7136b9c3-8b01-4279-9dbc-b37a92db6317","Type":"ContainerStarted","Data":"e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2"} Jul 06 14:49:41.884729 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:41.884495 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:41.934755 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:41.934702 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" podStartSLOduration=1.574066288 podStartE2EDuration="1.934686599s" totalImagesPullingTime="360.620311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:41.932007925 +0000 UTC m=+1517.474695455" watchObservedRunningTime="2026-07-06 14:49:41.934686599 +0000 UTC m=+1517.477374158" Jul 06 14:49:44.736995 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:44.736954 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz"] Jul 06 14:49:44.737545 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:44.737190 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" podUID="a61a2370-06f5-438c-ae6f-4d6d55fabbfa" containerName="istio-proxy" containerID="cri-o://68e0d8ab56c6694920c96b3c27c4a9076707b9f61fb5a1ca3f24ea07773912b9" gracePeriod=30 Jul 06 14:49:44.983130 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:44.983059 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv"] Jul 06 14:49:44.983349 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:44.983278 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" podUID="7136b9c3-8b01-4279-9dbc-b37a92db6317" containerName="maas-api" containerID="cri-o://e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2" gracePeriod=30 Jul 06 14:49:45.235887 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.235860 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:45.417239 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.417128 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7136b9c3-8b01-4279-9dbc-b37a92db6317-maas-api-tls\") pod \"7136b9c3-8b01-4279-9dbc-b37a92db6317\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " Jul 06 14:49:45.417239 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.417178 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7136b9c3-8b01-4279-9dbc-b37a92db6317-openshift-service-ca\") pod \"7136b9c3-8b01-4279-9dbc-b37a92db6317\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " Jul 06 14:49:45.417239 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.417213 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmzkw\" (UniqueName: \"kubernetes.io/projected/7136b9c3-8b01-4279-9dbc-b37a92db6317-kube-api-access-jmzkw\") pod \"7136b9c3-8b01-4279-9dbc-b37a92db6317\" (UID: \"7136b9c3-8b01-4279-9dbc-b37a92db6317\") " Jul 06 14:49:45.417592 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.417563 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7136b9c3-8b01-4279-9dbc-b37a92db6317-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7136b9c3-8b01-4279-9dbc-b37a92db6317" (UID: "7136b9c3-8b01-4279-9dbc-b37a92db6317"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:45.419483 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.419451 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7136b9c3-8b01-4279-9dbc-b37a92db6317-kube-api-access-jmzkw" (OuterVolumeSpecName: "kube-api-access-jmzkw") pod "7136b9c3-8b01-4279-9dbc-b37a92db6317" (UID: "7136b9c3-8b01-4279-9dbc-b37a92db6317"). InnerVolumeSpecName "kube-api-access-jmzkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:45.419483 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.419474 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7136b9c3-8b01-4279-9dbc-b37a92db6317-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7136b9c3-8b01-4279-9dbc-b37a92db6317" (UID: "7136b9c3-8b01-4279-9dbc-b37a92db6317"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:49:45.518601 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.518569 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmzkw\" (UniqueName: \"kubernetes.io/projected/7136b9c3-8b01-4279-9dbc-b37a92db6317-kube-api-access-jmzkw\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:45.518601 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.518599 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7136b9c3-8b01-4279-9dbc-b37a92db6317-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:45.518900 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.518608 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7136b9c3-8b01-4279-9dbc-b37a92db6317-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:45.904576 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.904532 2565 generic.go:356] "Generic (PLEG): container finished" podID="7136b9c3-8b01-4279-9dbc-b37a92db6317" containerID="e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2" exitCode=0 Jul 06 14:49:45.905000 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.904597 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" Jul 06 14:49:45.905000 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.904629 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" event={"ID":"7136b9c3-8b01-4279-9dbc-b37a92db6317","Type":"ContainerDied","Data":"e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2"} Jul 06 14:49:45.905000 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.904716 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv" event={"ID":"7136b9c3-8b01-4279-9dbc-b37a92db6317","Type":"ContainerDied","Data":"9a734a599ff72fe319d9bb96204732f8475f1fc97cf52b83befb5d172816c8ea"} Jul 06 14:49:45.905000 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.904740 2565 scope.go:117] "RemoveContainer" containerID="e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2" Jul 06 14:49:45.914548 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.914526 2565 scope.go:117] "RemoveContainer" containerID="e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2" Jul 06 14:49:45.914854 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:49:45.914833 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2\": container with ID starting with e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2 not found: ID does not exist" containerID="e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2" Jul 06 14:49:45.914927 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.914867 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2"} err="failed to get container status \"e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2\": rpc error: code = NotFound desc = could not find container \"e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2\": container with ID starting with e1e1fc008e7254652b1f2f7a2bbd4f84387d5d54b790b097ef08e7d42fa09fb2 not found: ID does not exist" Jul 06 14:49:45.933245 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.933221 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv"] Jul 06 14:49:45.940745 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:45.940720 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-9c941945-d748787bf-bg6zv"] Jul 06 14:49:46.932634 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:46.932593 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7136b9c3-8b01-4279-9dbc-b37a92db6317" path="/var/lib/kubelet/pods/7136b9c3-8b01-4279-9dbc-b37a92db6317/volumes" Jul 06 14:49:49.925363 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:49.925334 2565 generic.go:356] "Generic (PLEG): container finished" podID="a61a2370-06f5-438c-ae6f-4d6d55fabbfa" containerID="68e0d8ab56c6694920c96b3c27c4a9076707b9f61fb5a1ca3f24ea07773912b9" exitCode=0 Jul 06 14:49:49.925705 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:49.925367 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" event={"ID":"a61a2370-06f5-438c-ae6f-4d6d55fabbfa","Type":"ContainerDied","Data":"68e0d8ab56c6694920c96b3c27c4a9076707b9f61fb5a1ca3f24ea07773912b9"} Jul 06 14:49:49.988197 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:49.988171 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:50.060380 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060288 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-token\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060380 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060346 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-data\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060606 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060389 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-socket\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060606 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060418 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-certs\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060606 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060476 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-credential-socket\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060606 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060503 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istiod-ca-cert\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060606 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060542 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6fzm\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-kube-api-access-z6fzm\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060606 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060573 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-envoy\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060948 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060615 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-podinfo\") pod \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\" (UID: \"a61a2370-06f5-438c-ae6f-4d6d55fabbfa\") " Jul 06 14:49:50.060948 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060664 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:50.060948 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060673 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-data" (OuterVolumeSpecName: "istio-data") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:50.060948 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060776 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:50.060948 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060818 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:50.061181 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.060973 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:50.061181 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.061000 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-data\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.061181 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.061022 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.061181 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.061040 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-workload-certs\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.061181 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.061059 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-credential-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.062966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.062942 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:49:50.063066 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.062975 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-kube-api-access-z6fzm" (OuterVolumeSpecName: "kube-api-access-z6fzm") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "kube-api-access-z6fzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:50.063153 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.063138 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:50.063285 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.063268 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-token" (OuterVolumeSpecName: "istio-token") pod "a61a2370-06f5-438c-ae6f-4d6d55fabbfa" (UID: "a61a2370-06f5-438c-ae6f-4d6d55fabbfa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:50.161907 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.161870 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-envoy\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.161907 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.161896 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-podinfo\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.161907 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.161905 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istio-token\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.161907 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.161913 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-istiod-ca-cert\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.162178 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.161921 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6fzm\" (UniqueName: \"kubernetes.io/projected/a61a2370-06f5-438c-ae6f-4d6d55fabbfa-kube-api-access-z6fzm\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:49:50.932930 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.932902 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" Jul 06 14:49:50.933391 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.933368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz" event={"ID":"a61a2370-06f5-438c-ae6f-4d6d55fabbfa","Type":"ContainerDied","Data":"5d1954604e1e9ed7628e3caea5f2c0e95622f7fe1cef78e4ff27c156ffddd568"} Jul 06 14:49:50.933437 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.933417 2565 scope.go:117] "RemoveContainer" containerID="68e0d8ab56c6694920c96b3c27c4a9076707b9f61fb5a1ca3f24ea07773912b9" Jul 06 14:49:50.962383 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.962343 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz"] Jul 06 14:49:50.973414 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:50.973378 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9c941945-openshift-default-6669b9c7c9-522pz"] Jul 06 14:49:52.933583 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:49:52.933550 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a61a2370-06f5-438c-ae6f-4d6d55fabbfa" path="/var/lib/kubelet/pods/a61a2370-06f5-438c-ae6f-4d6d55fabbfa/volumes" Jul 06 14:51:36.715136 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.715055 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc"] Jul 06 14:51:36.715629 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.715614 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7136b9c3-8b01-4279-9dbc-b37a92db6317" containerName="maas-api" Jul 06 14:51:36.715677 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.715633 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7136b9c3-8b01-4279-9dbc-b37a92db6317" containerName="maas-api" Jul 06 14:51:36.715677 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.715650 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a61a2370-06f5-438c-ae6f-4d6d55fabbfa" containerName="istio-proxy" Jul 06 14:51:36.715677 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.715659 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a61a2370-06f5-438c-ae6f-4d6d55fabbfa" containerName="istio-proxy" Jul 06 14:51:36.715768 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.715736 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7136b9c3-8b01-4279-9dbc-b37a92db6317" containerName="maas-api" Jul 06 14:51:36.715768 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.715750 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a61a2370-06f5-438c-ae6f-4d6d55fabbfa" containerName="istio-proxy" Jul 06 14:51:36.719073 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.719057 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.723080 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.723058 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-95005850-serving-cert\"" Jul 06 14:51:36.733438 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.733419 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc"] Jul 06 14:51:36.807026 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.806982 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b93e101-cba3-480a-b2a6-0d172a336565-openshift-service-ca\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.807026 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.807031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffz7k\" (UniqueName: \"kubernetes.io/projected/3b93e101-cba3-480a-b2a6-0d172a336565-kube-api-access-ffz7k\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.807260 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.807070 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b93e101-cba3-480a-b2a6-0d172a336565-maas-api-tls\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.908143 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.908105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b93e101-cba3-480a-b2a6-0d172a336565-openshift-service-ca\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.908143 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.908148 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffz7k\" (UniqueName: \"kubernetes.io/projected/3b93e101-cba3-480a-b2a6-0d172a336565-kube-api-access-ffz7k\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.908371 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.908178 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b93e101-cba3-480a-b2a6-0d172a336565-maas-api-tls\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.908875 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.908856 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b93e101-cba3-480a-b2a6-0d172a336565-openshift-service-ca\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.910753 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.910738 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b93e101-cba3-480a-b2a6-0d172a336565-maas-api-tls\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:36.924049 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:36.924023 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffz7k\" (UniqueName: \"kubernetes.io/projected/3b93e101-cba3-480a-b2a6-0d172a336565-kube-api-access-ffz7k\") pod \"maas-api-e2e-mt-95005850-8477848fcc-jqmnc\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:37.032566 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:37.032479 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:37.172956 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:37.172927 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc"] Jul 06 14:51:37.174948 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:51:37.174922 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b93e101_cba3_480a_b2a6_0d172a336565.slice/crio-5ce1e0100ee684e29337ec13abe743b04edba6b566e9792b7b7200bb77d0ffad WatchSource:0}: Error finding container 5ce1e0100ee684e29337ec13abe743b04edba6b566e9792b7b7200bb77d0ffad: Status 404 returned error can't find the container with id 5ce1e0100ee684e29337ec13abe743b04edba6b566e9792b7b7200bb77d0ffad Jul 06 14:51:37.427737 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:37.427638 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" event={"ID":"3b93e101-cba3-480a-b2a6-0d172a336565","Type":"ContainerStarted","Data":"5ce1e0100ee684e29337ec13abe743b04edba6b566e9792b7b7200bb77d0ffad"} Jul 06 14:51:38.433394 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:38.433359 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" event={"ID":"3b93e101-cba3-480a-b2a6-0d172a336565","Type":"ContainerStarted","Data":"19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21"} Jul 06 14:51:38.433811 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:38.433419 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:38.458323 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:38.458262 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" podStartSLOduration=1.911975343 podStartE2EDuration="2.458240316s" totalImagesPullingTime="546.264973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:51:38.456706659 +0000 UTC m=+1633.999394197" watchObservedRunningTime="2026-07-06 14:51:38.458240316 +0000 UTC m=+1634.000927856" Jul 06 14:51:44.443977 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:44.443939 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:49.918906 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:49.918873 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc"] Jul 06 14:51:49.919296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:49.919080 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" podUID="3b93e101-cba3-480a-b2a6-0d172a336565" containerName="maas-api" containerID="cri-o://19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21" gracePeriod=30 Jul 06 14:51:50.170109 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.170043 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:50.228640 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.228611 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b93e101-cba3-480a-b2a6-0d172a336565-maas-api-tls\") pod \"3b93e101-cba3-480a-b2a6-0d172a336565\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " Jul 06 14:51:50.228855 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.228656 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b93e101-cba3-480a-b2a6-0d172a336565-openshift-service-ca\") pod \"3b93e101-cba3-480a-b2a6-0d172a336565\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " Jul 06 14:51:50.228855 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.228712 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffz7k\" (UniqueName: \"kubernetes.io/projected/3b93e101-cba3-480a-b2a6-0d172a336565-kube-api-access-ffz7k\") pod \"3b93e101-cba3-480a-b2a6-0d172a336565\" (UID: \"3b93e101-cba3-480a-b2a6-0d172a336565\") " Jul 06 14:51:50.229097 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.229068 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b93e101-cba3-480a-b2a6-0d172a336565-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b93e101-cba3-480a-b2a6-0d172a336565" (UID: "3b93e101-cba3-480a-b2a6-0d172a336565"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:51:50.231003 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.230972 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b93e101-cba3-480a-b2a6-0d172a336565-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b93e101-cba3-480a-b2a6-0d172a336565" (UID: "3b93e101-cba3-480a-b2a6-0d172a336565"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:51:50.231099 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.231016 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b93e101-cba3-480a-b2a6-0d172a336565-kube-api-access-ffz7k" (OuterVolumeSpecName: "kube-api-access-ffz7k") pod "3b93e101-cba3-480a-b2a6-0d172a336565" (UID: "3b93e101-cba3-480a-b2a6-0d172a336565"). InnerVolumeSpecName "kube-api-access-ffz7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:51:50.329433 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.329395 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b93e101-cba3-480a-b2a6-0d172a336565-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:51:50.329433 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.329426 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b93e101-cba3-480a-b2a6-0d172a336565-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:51:50.329433 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.329441 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffz7k\" (UniqueName: \"kubernetes.io/projected/3b93e101-cba3-480a-b2a6-0d172a336565-kube-api-access-ffz7k\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:51:50.491761 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.491723 2565 generic.go:356] "Generic (PLEG): container finished" podID="3b93e101-cba3-480a-b2a6-0d172a336565" containerID="19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21" exitCode=0 Jul 06 14:51:50.491971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.491771 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" Jul 06 14:51:50.491971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.491805 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" event={"ID":"3b93e101-cba3-480a-b2a6-0d172a336565","Type":"ContainerDied","Data":"19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21"} Jul 06 14:51:50.491971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.491849 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc" event={"ID":"3b93e101-cba3-480a-b2a6-0d172a336565","Type":"ContainerDied","Data":"5ce1e0100ee684e29337ec13abe743b04edba6b566e9792b7b7200bb77d0ffad"} Jul 06 14:51:50.491971 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.491871 2565 scope.go:117] "RemoveContainer" containerID="19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21" Jul 06 14:51:50.501070 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.501048 2565 scope.go:117] "RemoveContainer" containerID="19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21" Jul 06 14:51:50.501316 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:51:50.501299 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21\": container with ID starting with 19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21 not found: ID does not exist" containerID="19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21" Jul 06 14:51:50.501358 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.501324 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21"} err="failed to get container status \"19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21\": rpc error: code = NotFound desc = could not find container \"19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21\": container with ID starting with 19f510484416efc08b523a0c1fb3284f581d3eb34924b3d444a3cd6a2ee63f21 not found: ID does not exist" Jul 06 14:51:50.516736 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.516705 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc"] Jul 06 14:51:50.521631 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.521609 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-95005850-8477848fcc-jqmnc"] Jul 06 14:51:50.933574 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:51:50.933490 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b93e101-cba3-480a-b2a6-0d172a336565" path="/var/lib/kubelet/pods/3b93e101-cba3-480a-b2a6-0d172a336565/volumes" Jul 06 14:52:17.228650 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.228610 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w"] Jul 06 14:52:17.229232 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.229212 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b93e101-cba3-480a-b2a6-0d172a336565" containerName="maas-api" Jul 06 14:52:17.229305 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.229237 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b93e101-cba3-480a-b2a6-0d172a336565" containerName="maas-api" Jul 06 14:52:17.229362 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.229340 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b93e101-cba3-480a-b2a6-0d172a336565" containerName="maas-api" Jul 06 14:52:17.232371 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.232352 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.258029 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.257993 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-277e79e2-openshift-default-dockercfg-czqld\"" Jul 06 14:52:17.267858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.267829 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w"] Jul 06 14:52:17.368422 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-data\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.368643 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368434 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-token\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.368643 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-socket\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.368643 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368605 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxtkf\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-kube-api-access-zxtkf\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.368830 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368664 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-envoy\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.368830 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368728 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-credential-socket\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.368830 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368762 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-certs\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.368830 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368801 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-podinfo\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.369015 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.368864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bfac690-7d89-47d8-ab64-4df969f58fca-istiod-ca-cert\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.469470 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-socket\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.469675 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469500 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxtkf\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-kube-api-access-zxtkf\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.469675 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-envoy\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.469675 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-credential-socket\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.469675 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469618 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-certs\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.469675 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469644 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-podinfo\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.469675 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469673 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bfac690-7d89-47d8-ab64-4df969f58fca-istiod-ca-cert\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.470025 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469711 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-data\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.470025 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469740 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-token\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.470025 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.469904 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-socket\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.470443 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.470420 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-credential-socket\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.470576 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.470527 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-certs\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.470576 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.470542 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-data\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.470576 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.470494 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bfac690-7d89-47d8-ab64-4df969f58fca-istiod-ca-cert\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.472494 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.472464 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-podinfo\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.472586 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.472506 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-envoy\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.481143 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.481076 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-token\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.481143 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.481089 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxtkf\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-kube-api-access-zxtkf\") pod \"e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.546422 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.546388 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:17.699383 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.699360 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w"] Jul 06 14:52:17.701391 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:52:17.701357 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bfac690_7d89_47d8_ab64_4df969f58fca.slice/crio-469f111b10d62e96b3620f6ae06f72f78d801c7f4c2ed884abccb6725f55b236 WatchSource:0}: Error finding container 469f111b10d62e96b3620f6ae06f72f78d801c7f4c2ed884abccb6725f55b236: Status 404 returned error can't find the container with id 469f111b10d62e96b3620f6ae06f72f78d801c7f4c2ed884abccb6725f55b236 Jul 06 14:52:17.703496 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.703463 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:52:17.703593 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.703531 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:52:17.703593 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:17.703559 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:52:18.630252 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:18.630210 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" event={"ID":"3bfac690-7d89-47d8-ab64-4df969f58fca","Type":"ContainerStarted","Data":"bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f"} Jul 06 14:52:18.630622 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:18.630267 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" event={"ID":"3bfac690-7d89-47d8-ab64-4df969f58fca","Type":"ContainerStarted","Data":"469f111b10d62e96b3620f6ae06f72f78d801c7f4c2ed884abccb6725f55b236"} Jul 06 14:52:18.661433 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:18.661368 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" podStartSLOduration=1.6613441519999999 podStartE2EDuration="1.661344152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:52:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:52:18.659250556 +0000 UTC m=+1674.201938096" watchObservedRunningTime="2026-07-06 14:52:18.661344152 +0000 UTC m=+1674.204031691" Jul 06 14:52:19.547406 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:19.547363 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:19.552433 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:19.552406 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:19.635402 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:19.635366 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:19.636326 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:19.636305 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:20.878526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:20.878493 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd"] Jul 06 14:52:20.882518 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:20.882498 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:20.885098 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:20.885077 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-277e79e2-serving-cert\"" Jul 06 14:52:20.894307 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:20.894275 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd"] Jul 06 14:52:20.907591 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:20.907566 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67215880-0ac0-4c51-b375-428987fdb087-openshift-service-ca\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:20.907731 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:20.907603 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:20.907731 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:20.907639 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j84pw\" (UniqueName: \"kubernetes.io/projected/67215880-0ac0-4c51-b375-428987fdb087-kube-api-access-j84pw\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.008121 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.008087 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67215880-0ac0-4c51-b375-428987fdb087-openshift-service-ca\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.008306 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.008138 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.008306 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.008181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j84pw\" (UniqueName: \"kubernetes.io/projected/67215880-0ac0-4c51-b375-428987fdb087-kube-api-access-j84pw\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.008306 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:52:21.008267 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-277e79e2-serving-cert: secret "maas-api-e2e-mt-277e79e2-serving-cert" not found Jul 06 14:52:21.008496 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:52:21.008353 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls podName:67215880-0ac0-4c51-b375-428987fdb087 nodeName:}" failed. No retries permitted until 2026-07-06 14:52:21.508330892 +0000 UTC m=+1677.051018410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls") pod "maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" (UID: "67215880-0ac0-4c51-b375-428987fdb087") : secret "maas-api-e2e-mt-277e79e2-serving-cert" not found Jul 06 14:52:21.008868 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.008846 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67215880-0ac0-4c51-b375-428987fdb087-openshift-service-ca\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.017655 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.017632 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j84pw\" (UniqueName: \"kubernetes.io/projected/67215880-0ac0-4c51-b375-428987fdb087-kube-api-access-j84pw\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.513382 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.513349 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.515901 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.515882 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls\") pod \"maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.796744 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.796633 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:21.936939 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:21.936913 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd"] Jul 06 14:52:21.939140 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:52:21.939112 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67215880_0ac0_4c51_b375_428987fdb087.slice/crio-e4a605c4954740dc66fd2355735974d10fa97e125d468ecd20b3baa1f87830cc WatchSource:0}: Error finding container e4a605c4954740dc66fd2355735974d10fa97e125d468ecd20b3baa1f87830cc: Status 404 returned error can't find the container with id e4a605c4954740dc66fd2355735974d10fa97e125d468ecd20b3baa1f87830cc Jul 06 14:52:22.652933 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:22.652828 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" event={"ID":"67215880-0ac0-4c51-b375-428987fdb087","Type":"ContainerStarted","Data":"ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55"} Jul 06 14:52:22.652933 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:22.652898 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" event={"ID":"67215880-0ac0-4c51-b375-428987fdb087","Type":"ContainerStarted","Data":"e4a605c4954740dc66fd2355735974d10fa97e125d468ecd20b3baa1f87830cc"} Jul 06 14:52:22.652933 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:22.652926 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:22.672604 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:22.672554 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" podStartSLOduration=2.346144881 podStartE2EDuration="2.672537795s" totalImagesPullingTime="326.392914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:52:22.671179767 +0000 UTC m=+1678.213867306" watchObservedRunningTime="2026-07-06 14:52:22.672537795 +0000 UTC m=+1678.215225334" Jul 06 14:52:28.672447 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:28.672394 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:52:33.711839 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:33.711803 2565 generic.go:356] "Generic (PLEG): container finished" podID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerID="bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f" exitCode=0 Jul 06 14:52:33.712257 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:33.711858 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" event={"ID":"3bfac690-7d89-47d8-ab64-4df969f58fca","Type":"ContainerDied","Data":"bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f"} Jul 06 14:52:33.712301 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:33.712271 2565 scope.go:117] "RemoveContainer" containerID="bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f" Jul 06 14:52:33.713328 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:33.713297 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:52:33.713403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:33.713386 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:52:33.713453 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:33.713438 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:52:34.635692 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:34.635657 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:34.719368 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:34.719330 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" event={"ID":"3bfac690-7d89-47d8-ab64-4df969f58fca","Type":"ContainerStarted","Data":"6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc"} Jul 06 14:52:35.547312 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:35.547272 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:35.548631 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:35.548609 2565 patch_prober.go:28] interesting pod/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 06 14:52:35.548700 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:35.548655 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 06 14:52:36.547611 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:36.547574 2565 patch_prober.go:28] interesting pod/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 06 14:52:36.548029 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:36.547631 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 06 14:52:37.547610 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:37.547569 2565 patch_prober.go:28] interesting pod/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 06 14:52:37.547857 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:37.547638 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 06 14:52:38.547472 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:38.547435 2565 patch_prober.go:28] interesting pod/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 06 14:52:38.547660 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:38.547490 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 06 14:52:39.551449 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:39.551418 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:39.551865 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:39.551713 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:52:39.552399 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:52:39.552383 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:53:13.538762 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.538730 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd"] Jul 06 14:53:13.539380 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.538991 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" podUID="67215880-0ac0-4c51-b375-428987fdb087" containerName="maas-api" containerID="cri-o://ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55" gracePeriod=30 Jul 06 14:53:13.660013 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.659968 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" podUID="67215880-0ac0-4c51-b375-428987fdb087" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.51:8443/health\": dial tcp 10.133.0.51:8443: connect: connection refused" Jul 06 14:53:13.797304 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.797238 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:53:13.897077 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.897034 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls\") pod \"67215880-0ac0-4c51-b375-428987fdb087\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " Jul 06 14:53:13.897285 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.897140 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j84pw\" (UniqueName: \"kubernetes.io/projected/67215880-0ac0-4c51-b375-428987fdb087-kube-api-access-j84pw\") pod \"67215880-0ac0-4c51-b375-428987fdb087\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " Jul 06 14:53:13.897285 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.897165 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67215880-0ac0-4c51-b375-428987fdb087-openshift-service-ca\") pod \"67215880-0ac0-4c51-b375-428987fdb087\" (UID: \"67215880-0ac0-4c51-b375-428987fdb087\") " Jul 06 14:53:13.897579 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.897555 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67215880-0ac0-4c51-b375-428987fdb087-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67215880-0ac0-4c51-b375-428987fdb087" (UID: "67215880-0ac0-4c51-b375-428987fdb087"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:53:13.899276 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.899252 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67215880-0ac0-4c51-b375-428987fdb087" (UID: "67215880-0ac0-4c51-b375-428987fdb087"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:53:13.899407 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.899383 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67215880-0ac0-4c51-b375-428987fdb087-kube-api-access-j84pw" (OuterVolumeSpecName: "kube-api-access-j84pw") pod "67215880-0ac0-4c51-b375-428987fdb087" (UID: "67215880-0ac0-4c51-b375-428987fdb087"). InnerVolumeSpecName "kube-api-access-j84pw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:53:13.910094 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.910067 2565 generic.go:356] "Generic (PLEG): container finished" podID="67215880-0ac0-4c51-b375-428987fdb087" containerID="ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55" exitCode=0 Jul 06 14:53:13.910224 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.910115 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" Jul 06 14:53:13.910224 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.910113 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" event={"ID":"67215880-0ac0-4c51-b375-428987fdb087","Type":"ContainerDied","Data":"ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55"} Jul 06 14:53:13.910224 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.910183 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd" event={"ID":"67215880-0ac0-4c51-b375-428987fdb087","Type":"ContainerDied","Data":"e4a605c4954740dc66fd2355735974d10fa97e125d468ecd20b3baa1f87830cc"} Jul 06 14:53:13.910224 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.910213 2565 scope.go:117] "RemoveContainer" containerID="ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55" Jul 06 14:53:13.920004 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.919981 2565 scope.go:117] "RemoveContainer" containerID="ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55" Jul 06 14:53:13.920320 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:53:13.920299 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55\": container with ID starting with ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55 not found: ID does not exist" containerID="ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55" Jul 06 14:53:13.920389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.920328 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55"} err="failed to get container status \"ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55\": rpc error: code = NotFound desc = could not find container \"ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55\": container with ID starting with ac6bf848077c82283408d6e925dce40bdee21cc02d74810ee4b4f8f0d3165e55 not found: ID does not exist" Jul 06 14:53:13.933317 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.933288 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd"] Jul 06 14:53:13.937870 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.937844 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-277e79e2-b9f6db9c5-t4ghd"] Jul 06 14:53:13.998720 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.998680 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67215880-0ac0-4c51-b375-428987fdb087-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:53:13.998720 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.998713 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j84pw\" (UniqueName: \"kubernetes.io/projected/67215880-0ac0-4c51-b375-428987fdb087-kube-api-access-j84pw\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:53:13.998720 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:13.998723 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67215880-0ac0-4c51-b375-428987fdb087-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:53:14.932657 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:53:14.932625 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67215880-0ac0-4c51-b375-428987fdb087" path="/var/lib/kubelet/pods/67215880-0ac0-4c51-b375-428987fdb087/volumes" Jul 06 14:54:23.755106 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:23.755068 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w"] Jul 06 14:54:23.755599 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:23.755341 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" containerID="cri-o://6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc" gracePeriod=30 Jul 06 14:54:24.554663 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:24.554622 2565 patch_prober.go:28] interesting pod/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:54:24.554862 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:24.554688 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:54:29.026201 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.026131 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:54:29.189876 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.189845 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-socket\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.189876 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.189879 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-certs\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.190139 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.189909 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-data\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.190139 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.189931 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-podinfo\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.190139 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190001 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxtkf\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-kube-api-access-zxtkf\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.190139 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190032 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-envoy\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.190343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190110 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-token\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.190403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190342 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bfac690-7d89-47d8-ab64-4df969f58fca-istiod-ca-cert\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.190403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190230 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:54:29.190521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190265 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:54:29.190521 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190429 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-credential-socket\") pod \"3bfac690-7d89-47d8-ab64-4df969f58fca\" (UID: \"3bfac690-7d89-47d8-ab64-4df969f58fca\") " Jul 06 14:54:29.190714 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190688 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:54:29.191002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190832 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-credential-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.191002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190852 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.191002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190866 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-workload-certs\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.191002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.190920 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-data" (OuterVolumeSpecName: "istio-data") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:54:29.191227 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.191015 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bfac690-7d89-47d8-ab64-4df969f58fca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:54:29.192552 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.192521 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:54:29.192552 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.192525 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:54:29.192718 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.192521 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-kube-api-access-zxtkf" (OuterVolumeSpecName: "kube-api-access-zxtkf") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "kube-api-access-zxtkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:54:29.192718 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.192638 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-token" (OuterVolumeSpecName: "istio-token") pod "3bfac690-7d89-47d8-ab64-4df969f58fca" (UID: "3bfac690-7d89-47d8-ab64-4df969f58fca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:54:29.273429 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.273394 2565 generic.go:356] "Generic (PLEG): container finished" podID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerID="6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc" exitCode=0 Jul 06 14:54:29.273627 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.273443 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" Jul 06 14:54:29.273627 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.273465 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" event={"ID":"3bfac690-7d89-47d8-ab64-4df969f58fca","Type":"ContainerDied","Data":"6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc"} Jul 06 14:54:29.273627 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.273520 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w" event={"ID":"3bfac690-7d89-47d8-ab64-4df969f58fca","Type":"ContainerDied","Data":"469f111b10d62e96b3620f6ae06f72f78d801c7f4c2ed884abccb6725f55b236"} Jul 06 14:54:29.273627 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.273548 2565 scope.go:117] "RemoveContainer" containerID="6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc" Jul 06 14:54:29.283174 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.283156 2565 scope.go:117] "RemoveContainer" containerID="bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f" Jul 06 14:54:29.291458 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.291431 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-data\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.291566 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.291464 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-podinfo\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.291566 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.291479 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxtkf\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-kube-api-access-zxtkf\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.291566 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.291493 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-envoy\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.291566 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.291507 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bfac690-7d89-47d8-ab64-4df969f58fca-istio-token\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.291566 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.291519 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bfac690-7d89-47d8-ab64-4df969f58fca-istiod-ca-cert\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:54:29.292241 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.292214 2565 scope.go:117] "RemoveContainer" containerID="6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc" Jul 06 14:54:29.292499 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:54:29.292482 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc\": container with ID starting with 6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc not found: ID does not exist" containerID="6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc" Jul 06 14:54:29.292555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.292509 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc"} err="failed to get container status \"6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc\": rpc error: code = NotFound desc = could not find container \"6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc\": container with ID starting with 6cde7ea5cda6d3d99458e2d868dd4f4eaee60c4fe63406ad6ff9f98a08d58bbc not found: ID does not exist" Jul 06 14:54:29.292555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.292532 2565 scope.go:117] "RemoveContainer" containerID="bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f" Jul 06 14:54:29.292768 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:54:29.292751 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f\": container with ID starting with bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f not found: ID does not exist" containerID="bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f" Jul 06 14:54:29.292843 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.292774 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f"} err="failed to get container status \"bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f\": rpc error: code = NotFound desc = could not find container \"bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f\": container with ID starting with bf2172ee112c8f6e28d779d5920d0dcc6aafc52db60aa385027a48088d55532f not found: ID does not exist" Jul 06 14:54:29.299732 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.299689 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w"] Jul 06 14:54:29.306961 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:29.306921 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-277e79e2-openshift-default-7f6bd89d87-77g6w"] Jul 06 14:54:30.932377 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:54:30.932339 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" path="/var/lib/kubelet/pods/3bfac690-7d89-47d8-ab64-4df969f58fca/volumes" Jul 06 14:55:19.581145 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.581106 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t"] Jul 06 14:55:19.581642 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.581502 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" Jul 06 14:55:19.581642 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.581519 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" Jul 06 14:55:19.581642 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.581545 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67215880-0ac0-4c51-b375-428987fdb087" containerName="maas-api" Jul 06 14:55:19.581642 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.581550 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="67215880-0ac0-4c51-b375-428987fdb087" containerName="maas-api" Jul 06 14:55:19.581642 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.581610 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" Jul 06 14:55:19.581642 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.581623 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="67215880-0ac0-4c51-b375-428987fdb087" containerName="maas-api" Jul 06 14:55:19.584643 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.584624 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.587670 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.587648 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8e878a-openshift-default-dockercfg-jqkv8\"" Jul 06 14:55:19.596762 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.596722 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t"] Jul 06 14:55:19.737936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.737894 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-socket\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.737936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.737935 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdpfm\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-kube-api-access-tdpfm\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.738188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.738003 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-podinfo\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.738188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.738022 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-credential-socket\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.738188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.738041 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-certs\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.738188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.738078 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istiod-ca-cert\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.738188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.738125 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-data\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.738188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.738156 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-envoy\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.738188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.738186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-token\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.839736 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839636 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-data\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.839736 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839680 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-envoy\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.839736 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839711 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-token\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840045 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839750 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-socket\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840045 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdpfm\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-kube-api-access-tdpfm\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840045 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839846 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-podinfo\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840045 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-credential-socket\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840045 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839884 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-certs\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840045 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.839911 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istiod-ca-cert\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840357 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.840167 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-data\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840357 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.840235 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-socket\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840490 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.840467 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-certs\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840718 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.840472 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-credential-socket\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.840845 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.840805 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istiod-ca-cert\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.842278 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.842254 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-envoy\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.842476 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.842453 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-podinfo\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.849020 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.848997 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdpfm\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-kube-api-access-tdpfm\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.849143 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.849105 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-token\") pod \"e2e-inf-a-8e878a-openshift-default-56874fb555-j946t\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:19.898544 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:19.898506 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:20.049207 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:20.049178 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t"] Jul 06 14:55:20.051294 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:55:20.051270 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1648d32_c6ca_42c1_ac59_1ab0b50fa716.slice/crio-02469cc8bcaa2cc9eef9d073428a6d2dacfdfc54d0d448f80fab2b79d5a361e4 WatchSource:0}: Error finding container 02469cc8bcaa2cc9eef9d073428a6d2dacfdfc54d0d448f80fab2b79d5a361e4: Status 404 returned error can't find the container with id 02469cc8bcaa2cc9eef9d073428a6d2dacfdfc54d0d448f80fab2b79d5a361e4 Jul 06 14:55:20.054075 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:20.054029 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:55:20.054180 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:20.054130 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:55:20.054180 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:20.054173 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:55:20.529719 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:20.529678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" event={"ID":"c1648d32-c6ca-42c1-ac59-1ab0b50fa716","Type":"ContainerStarted","Data":"a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2"} Jul 06 14:55:20.529931 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:20.529743 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" event={"ID":"c1648d32-c6ca-42c1-ac59-1ab0b50fa716","Type":"ContainerStarted","Data":"02469cc8bcaa2cc9eef9d073428a6d2dacfdfc54d0d448f80fab2b79d5a361e4"} Jul 06 14:55:20.555842 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:20.555775 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" podStartSLOduration=1.555757995 podStartE2EDuration="1.555757995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:55:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:55:20.553185244 +0000 UTC m=+1856.095872782" watchObservedRunningTime="2026-07-06 14:55:20.555757995 +0000 UTC m=+1856.098445599" Jul 06 14:55:20.899512 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:20.899407 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:21.904035 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:21.904006 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:22.539352 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.539314 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:22.540340 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.540321 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:22.763842 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.763808 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx"] Jul 06 14:55:22.764204 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.764191 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" Jul 06 14:55:22.764258 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.764207 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" Jul 06 14:55:22.764293 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.764279 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bfac690-7d89-47d8-ab64-4df969f58fca" containerName="istio-proxy" Jul 06 14:55:22.767507 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.767491 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:22.769190 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.769157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:22.769328 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.769257 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:22.769328 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.769286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpvs9\" (UniqueName: \"kubernetes.io/projected/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-kube-api-access-kpvs9\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:22.770778 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.770759 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-8e878a-serving-cert\"" Jul 06 14:55:22.796080 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.796001 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx"] Jul 06 14:55:22.870639 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.870608 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:22.870866 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.870698 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:22.870866 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.870721 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpvs9\" (UniqueName: \"kubernetes.io/projected/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-kube-api-access-kpvs9\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:22.870866 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:55:22.870744 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-8e878a-serving-cert: secret "maas-api-e2e-inf-a-8e878a-serving-cert" not found Jul 06 14:55:22.870866 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:55:22.870835 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls podName:3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a nodeName:}" failed. No retries permitted until 2026-07-06 14:55:23.370818535 +0000 UTC m=+1858.913506051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls") pod "maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" (UID: "3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a") : secret "maas-api-e2e-inf-a-8e878a-serving-cert" not found Jul 06 14:55:22.871403 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.871383 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:22.882002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:22.881974 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpvs9\" (UniqueName: \"kubernetes.io/projected/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-kube-api-access-kpvs9\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:23.375136 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:23.375095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:23.377764 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:23.377743 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:23.378584 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:23.378558 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:23.528178 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:23.528151 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx"] Jul 06 14:55:23.530370 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:55:23.530330 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3366f4a8_5f8b_4e4c_8cd5_793948bf8d6a.slice/crio-ab727c58e608e3154df1d9150e078784376df8ffbe144460f4d296531d21ee4a WatchSource:0}: Error finding container ab727c58e608e3154df1d9150e078784376df8ffbe144460f4d296531d21ee4a: Status 404 returned error can't find the container with id ab727c58e608e3154df1d9150e078784376df8ffbe144460f4d296531d21ee4a Jul 06 14:55:23.531958 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:23.531929 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:55:23.544628 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:23.544586 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" event={"ID":"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a","Type":"ContainerStarted","Data":"ab727c58e608e3154df1d9150e078784376df8ffbe144460f4d296531d21ee4a"} Jul 06 14:55:24.550567 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:24.550528 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" event={"ID":"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a","Type":"ContainerStarted","Data":"b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d"} Jul 06 14:55:24.551015 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:24.550667 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:24.575647 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:24.575588 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" podStartSLOduration=2.136054019 podStartE2EDuration="2.575568368s" totalImagesPullingTime="439.514349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:55:24.57345742 +0000 UTC m=+1860.116144970" watchObservedRunningTime="2026-07-06 14:55:24.575568368 +0000 UTC m=+1860.118255905" Jul 06 14:55:30.562015 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:30.561985 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:55:30.869798 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:30.869698 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7"] Jul 06 14:55:30.875173 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:30.875146 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:30.878166 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:30.878139 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-5e9a43-serving-cert\"" Jul 06 14:55:30.884905 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:30.884877 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7"] Jul 06 14:55:30.942617 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:30.942581 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:30.942853 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:30.942623 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cc32f7-90d3-4382-8d00-47e540f8edb4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:30.942853 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:30.942688 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clzps\" (UniqueName: \"kubernetes.io/projected/c1cc32f7-90d3-4382-8d00-47e540f8edb4-kube-api-access-clzps\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.043701 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.043663 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cc32f7-90d3-4382-8d00-47e540f8edb4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.043922 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.043719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clzps\" (UniqueName: \"kubernetes.io/projected/c1cc32f7-90d3-4382-8d00-47e540f8edb4-kube-api-access-clzps\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.043922 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.043769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.043922 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:55:31.043880 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-5e9a43-serving-cert: secret "maas-api-e2e-inf-b-5e9a43-serving-cert" not found Jul 06 14:55:31.044105 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:55:31.043956 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls podName:c1cc32f7-90d3-4382-8d00-47e540f8edb4 nodeName:}" failed. No retries permitted until 2026-07-06 14:55:31.543929405 +0000 UTC m=+1867.086616925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls") pod "maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" (UID: "c1cc32f7-90d3-4382-8d00-47e540f8edb4") : secret "maas-api-e2e-inf-b-5e9a43-serving-cert" not found Jul 06 14:55:31.044440 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.044413 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cc32f7-90d3-4382-8d00-47e540f8edb4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.056906 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.056871 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clzps\" (UniqueName: \"kubernetes.io/projected/c1cc32f7-90d3-4382-8d00-47e540f8edb4-kube-api-access-clzps\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.548335 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.548304 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.551001 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.550973 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls\") pod \"maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.788738 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.788703 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:31.967827 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:31.965609 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7"] Jul 06 14:55:31.981067 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:55:31.980990 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1cc32f7_90d3_4382_8d00_47e540f8edb4.slice/crio-c8a38c275b97af97b223c451a85bdf75108da6614f56aed18245753699968974 WatchSource:0}: Error finding container c8a38c275b97af97b223c451a85bdf75108da6614f56aed18245753699968974: Status 404 returned error can't find the container with id c8a38c275b97af97b223c451a85bdf75108da6614f56aed18245753699968974 Jul 06 14:55:32.593509 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:32.593427 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" event={"ID":"c1cc32f7-90d3-4382-8d00-47e540f8edb4","Type":"ContainerStarted","Data":"26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da"} Jul 06 14:55:32.593509 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:32.593477 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" event={"ID":"c1cc32f7-90d3-4382-8d00-47e540f8edb4","Type":"ContainerStarted","Data":"c8a38c275b97af97b223c451a85bdf75108da6614f56aed18245753699968974"} Jul 06 14:55:32.593509 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:32.593510 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:32.611794 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:32.611732 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" podStartSLOduration=2.292083363 podStartE2EDuration="2.611710481s" totalImagesPullingTime="319.627118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:55:32.611120591 +0000 UTC m=+1868.153808128" watchObservedRunningTime="2026-07-06 14:55:32.611710481 +0000 UTC m=+1868.154398019" Jul 06 14:55:38.406310 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.406275 2565 patch_prober.go:28] interesting pod/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:35676->10.133.0.52:15021: read: connection reset by peer" start-of-body= Jul 06 14:55:38.406954 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.406353 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:35676->10.133.0.52:15021: read: connection reset by peer" Jul 06 14:55:38.604159 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.604087 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:55:38.624083 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.624048 2565 generic.go:356] "Generic (PLEG): container finished" podID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerID="a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2" exitCode=0 Jul 06 14:55:38.624248 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.624126 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" event={"ID":"c1648d32-c6ca-42c1-ac59-1ab0b50fa716","Type":"ContainerDied","Data":"a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2"} Jul 06 14:55:38.624689 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.624668 2565 scope.go:117] "RemoveContainer" containerID="a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2" Jul 06 14:55:38.626085 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.626053 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:55:38.626160 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.626131 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:55:38.626199 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:38.626167 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:55:39.635206 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:39.635165 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" event={"ID":"c1648d32-c6ca-42c1-ac59-1ab0b50fa716","Type":"ContainerStarted","Data":"b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6"} Jul 06 14:55:39.899206 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:39.899114 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:39.899371 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:39.899347 2565 patch_prober.go:28] interesting pod/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 06 14:55:39.899432 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:39.899410 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 06 14:55:40.899319 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:40.899279 2565 patch_prober.go:28] interesting pod/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 06 14:55:40.899721 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:40.899338 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 06 14:55:41.899890 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:41.899842 2565 patch_prober.go:28] interesting pod/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 06 14:55:41.900389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:41.899980 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 06 14:55:42.899543 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:42.899503 2565 patch_prober.go:28] interesting pod/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 06 14:55:42.899722 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:42.899558 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 06 14:55:43.902589 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:43.902559 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:43.903025 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:43.902827 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:55:43.903586 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:55:43.903570 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:57:17.161868 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.161739 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx"] Jul 06 14:57:17.162452 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.162029 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" podUID="3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" containerName="maas-api" containerID="cri-o://b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d" gracePeriod=30 Jul 06 14:57:17.415213 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.415147 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:57:17.581880 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.581841 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls\") pod \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " Jul 06 14:57:17.582090 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.581899 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-openshift-service-ca\") pod \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " Jul 06 14:57:17.582090 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.581940 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpvs9\" (UniqueName: \"kubernetes.io/projected/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-kube-api-access-kpvs9\") pod \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\" (UID: \"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a\") " Jul 06 14:57:17.582381 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.582352 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" (UID: "3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:57:17.584417 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.584387 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-kube-api-access-kpvs9" (OuterVolumeSpecName: "kube-api-access-kpvs9") pod "3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" (UID: "3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a"). InnerVolumeSpecName "kube-api-access-kpvs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:57:17.584525 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.584423 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" (UID: "3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:57:17.682868 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.682759 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpvs9\" (UniqueName: \"kubernetes.io/projected/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-kube-api-access-kpvs9\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:57:17.682868 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.682818 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:57:17.682868 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:17.682833 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:57:18.122137 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.122105 2565 generic.go:356] "Generic (PLEG): container finished" podID="3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" containerID="b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d" exitCode=0 Jul 06 14:57:18.122325 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.122149 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" Jul 06 14:57:18.122325 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.122186 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" event={"ID":"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a","Type":"ContainerDied","Data":"b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d"} Jul 06 14:57:18.122325 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.122238 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx" event={"ID":"3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a","Type":"ContainerDied","Data":"ab727c58e608e3154df1d9150e078784376df8ffbe144460f4d296531d21ee4a"} Jul 06 14:57:18.122325 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.122260 2565 scope.go:117] "RemoveContainer" containerID="b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d" Jul 06 14:57:18.132050 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.132033 2565 scope.go:117] "RemoveContainer" containerID="b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d" Jul 06 14:57:18.132310 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:57:18.132293 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d\": container with ID starting with b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d not found: ID does not exist" containerID="b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d" Jul 06 14:57:18.132380 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.132317 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d"} err="failed to get container status \"b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d\": rpc error: code = NotFound desc = could not find container \"b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d\": container with ID starting with b44fa9253125820aa5b0ce2c39235eb2e011832440d199ec959664e5a9822a4d not found: ID does not exist" Jul 06 14:57:18.145126 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.145093 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx"] Jul 06 14:57:18.149075 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.149050 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e878a-ccfd89986-dkcjx"] Jul 06 14:57:18.934246 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:57:18.934217 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" path="/var/lib/kubelet/pods/3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a/volumes" Jul 06 14:58:00.379326 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:00.379290 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t"] Jul 06 14:58:00.379925 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:00.379491 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" containerID="cri-o://b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6" gracePeriod=30 Jul 06 14:58:01.815501 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:01.815461 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7"] Jul 06 14:58:01.815931 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:01.815673 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" podUID="c1cc32f7-90d3-4382-8d00-47e540f8edb4" containerName="maas-api" containerID="cri-o://26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da" gracePeriod=30 Jul 06 14:58:02.065116 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.065086 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:58:02.200472 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.200416 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clzps\" (UniqueName: \"kubernetes.io/projected/c1cc32f7-90d3-4382-8d00-47e540f8edb4-kube-api-access-clzps\") pod \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " Jul 06 14:58:02.200662 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.200494 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cc32f7-90d3-4382-8d00-47e540f8edb4-openshift-service-ca\") pod \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " Jul 06 14:58:02.200662 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.200522 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls\") pod \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\" (UID: \"c1cc32f7-90d3-4382-8d00-47e540f8edb4\") " Jul 06 14:58:02.200910 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.200883 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1cc32f7-90d3-4382-8d00-47e540f8edb4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1cc32f7-90d3-4382-8d00-47e540f8edb4" (UID: "c1cc32f7-90d3-4382-8d00-47e540f8edb4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:58:02.202752 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.202730 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1cc32f7-90d3-4382-8d00-47e540f8edb4-kube-api-access-clzps" (OuterVolumeSpecName: "kube-api-access-clzps") pod "c1cc32f7-90d3-4382-8d00-47e540f8edb4" (UID: "c1cc32f7-90d3-4382-8d00-47e540f8edb4"). InnerVolumeSpecName "kube-api-access-clzps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:58:02.202853 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.202811 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1cc32f7-90d3-4382-8d00-47e540f8edb4" (UID: "c1cc32f7-90d3-4382-8d00-47e540f8edb4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:58:02.301745 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.301705 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cc32f7-90d3-4382-8d00-47e540f8edb4-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:02.301745 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.301739 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clzps\" (UniqueName: \"kubernetes.io/projected/c1cc32f7-90d3-4382-8d00-47e540f8edb4-kube-api-access-clzps\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:02.301745 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.301749 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cc32f7-90d3-4382-8d00-47e540f8edb4-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:02.335227 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.335191 2565 generic.go:356] "Generic (PLEG): container finished" podID="c1cc32f7-90d3-4382-8d00-47e540f8edb4" containerID="26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da" exitCode=0 Jul 06 14:58:02.335404 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.335254 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" event={"ID":"c1cc32f7-90d3-4382-8d00-47e540f8edb4","Type":"ContainerDied","Data":"26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da"} Jul 06 14:58:02.335404 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.335261 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" Jul 06 14:58:02.335404 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.335342 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7" event={"ID":"c1cc32f7-90d3-4382-8d00-47e540f8edb4","Type":"ContainerDied","Data":"c8a38c275b97af97b223c451a85bdf75108da6614f56aed18245753699968974"} Jul 06 14:58:02.335404 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.335374 2565 scope.go:117] "RemoveContainer" containerID="26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da" Jul 06 14:58:02.345258 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.345224 2565 scope.go:117] "RemoveContainer" containerID="26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da" Jul 06 14:58:02.345518 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:02.345494 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da\": container with ID starting with 26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da not found: ID does not exist" containerID="26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da" Jul 06 14:58:02.345584 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.345532 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da"} err="failed to get container status \"26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da\": rpc error: code = NotFound desc = could not find container \"26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da\": container with ID starting with 26e1ae54b2c7549837b01a816d25bd4de6ca0d68fdc9f414716e50bb6b13a6da not found: ID does not exist" Jul 06 14:58:02.361764 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.361728 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7"] Jul 06 14:58:02.364220 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.364197 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-5e9a43-565d7dd4bc-l7vj7"] Jul 06 14:58:02.933167 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:02.933134 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1cc32f7-90d3-4382-8d00-47e540f8edb4" path="/var/lib/kubelet/pods/c1cc32f7-90d3-4382-8d00-47e540f8edb4/volumes" Jul 06 14:58:05.638359 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.638333 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:58:05.735304 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735269 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istiod-ca-cert\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735326 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-data\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735350 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-certs\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735364 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-envoy\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735382 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-socket\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735410 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdpfm\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-kube-api-access-tdpfm\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735444 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-credential-socket\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735832 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735530 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-podinfo\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735832 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735556 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-token\") pod \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\" (UID: \"c1648d32-c6ca-42c1-ac59-1ab0b50fa716\") " Jul 06 14:58:05.735832 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735704 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:05.735989 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735837 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:05.735989 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735861 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:58:05.735989 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735871 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-certs\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:05.735989 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.735885 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:05.736426 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.736399 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:05.737699 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.737673 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:05.738052 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.738032 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-kube-api-access-tdpfm" (OuterVolumeSpecName: "kube-api-access-tdpfm") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "kube-api-access-tdpfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:58:05.738367 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.738346 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:58:05.738434 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.738348 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1648d32-c6ca-42c1-ac59-1ab0b50fa716" (UID: "c1648d32-c6ca-42c1-ac59-1ab0b50fa716"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:58:05.837093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.837055 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-token\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:05.837093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.837084 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istiod-ca-cert\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:05.837093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.837094 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-data\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:05.837093 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.837103 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-envoy\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:05.837370 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.837111 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-workload-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:05.837370 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.837120 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdpfm\" (UniqueName: \"kubernetes.io/projected/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-kube-api-access-tdpfm\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:05.837370 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.837130 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-credential-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:05.837370 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:05.837139 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1648d32-c6ca-42c1-ac59-1ab0b50fa716-istio-podinfo\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:06.358264 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.358225 2565 generic.go:356] "Generic (PLEG): container finished" podID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerID="b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6" exitCode=0 Jul 06 14:58:06.358454 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.358274 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" event={"ID":"c1648d32-c6ca-42c1-ac59-1ab0b50fa716","Type":"ContainerDied","Data":"b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6"} Jul 06 14:58:06.358454 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.358282 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" Jul 06 14:58:06.358454 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.358319 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t" event={"ID":"c1648d32-c6ca-42c1-ac59-1ab0b50fa716","Type":"ContainerDied","Data":"02469cc8bcaa2cc9eef9d073428a6d2dacfdfc54d0d448f80fab2b79d5a361e4"} Jul 06 14:58:06.358454 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.358340 2565 scope.go:117] "RemoveContainer" containerID="b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6" Jul 06 14:58:06.368201 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.368177 2565 scope.go:117] "RemoveContainer" containerID="a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2" Jul 06 14:58:06.377129 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.377108 2565 scope.go:117] "RemoveContainer" containerID="b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6" Jul 06 14:58:06.377400 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:06.377380 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6\": container with ID starting with b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6 not found: ID does not exist" containerID="b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6" Jul 06 14:58:06.377475 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.377413 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6"} err="failed to get container status \"b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6\": rpc error: code = NotFound desc = could not find container \"b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6\": container with ID starting with b5dbd087d0165d0de42161836e9498bcd89be024135aa0cb08e7e2cb34853ba6 not found: ID does not exist" Jul 06 14:58:06.377475 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.377440 2565 scope.go:117] "RemoveContainer" containerID="a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2" Jul 06 14:58:06.377698 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:06.377680 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2\": container with ID starting with a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2 not found: ID does not exist" containerID="a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2" Jul 06 14:58:06.377743 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.377704 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2"} err="failed to get container status \"a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2\": rpc error: code = NotFound desc = could not find container \"a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2\": container with ID starting with a7ea36f7993d32de59cf4d803139b495e500ff1ec7ddde60d50ec04d039bc6b2 not found: ID does not exist" Jul 06 14:58:06.382147 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.382123 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t"] Jul 06 14:58:06.386618 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.386595 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8e878a-openshift-default-56874fb555-j946t"] Jul 06 14:58:06.933992 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:06.933960 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" path="/var/lib/kubelet/pods/c1648d32-c6ca-42c1-ac59-1ab0b50fa716/volumes" Jul 06 14:58:09.315287 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.315254 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz"] Jul 06 14:58:09.315816 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.315778 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" Jul 06 14:58:09.315908 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.315821 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" Jul 06 14:58:09.315908 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.315843 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1cc32f7-90d3-4382-8d00-47e540f8edb4" containerName="maas-api" Jul 06 14:58:09.315908 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.315851 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1cc32f7-90d3-4382-8d00-47e540f8edb4" containerName="maas-api" Jul 06 14:58:09.315908 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.315886 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" containerName="maas-api" Jul 06 14:58:09.315908 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.315895 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" containerName="maas-api" Jul 06 14:58:09.316194 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.315997 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" Jul 06 14:58:09.316194 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.316010 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" Jul 06 14:58:09.316194 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.316022 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1cc32f7-90d3-4382-8d00-47e540f8edb4" containerName="maas-api" Jul 06 14:58:09.316194 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.316032 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3366f4a8-5f8b-4e4c-8cd5-793948bf8d6a" containerName="maas-api" Jul 06 14:58:09.320627 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.320605 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.325002 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.324980 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3b4c5e-openshift-default-dockercfg-4gcjh\"" Jul 06 14:58:09.333077 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.333055 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz"] Jul 06 14:58:09.365538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365499 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7047851e-547c-4216-ac94-50c4bd211f0a-istiod-ca-cert\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.365741 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365607 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-envoy\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.365741 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365651 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpvp2\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-kube-api-access-cpvp2\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.365741 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365710 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-istio-token\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.365919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365753 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-credential-socket\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.365919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365773 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-certs\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.365919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365827 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-data\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.365919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7047851e-547c-4216-ac94-50c4bd211f0a-istio-podinfo\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.365919 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.365891 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-socket\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.466936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.466903 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-socket\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.466958 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7047851e-547c-4216-ac94-50c4bd211f0a-istiod-ca-cert\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467000 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-envoy\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpvp2\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-kube-api-access-cpvp2\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467131 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467067 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-istio-token\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467201 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-credential-socket\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-certs\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467274 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-data\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7047851e-547c-4216-ac94-50c4bd211f0a-istio-podinfo\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467545 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467338 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-socket\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467545 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467522 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-credential-socket\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467709 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467685 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-certs\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467831 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467714 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-data\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.467831 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.467735 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7047851e-547c-4216-ac94-50c4bd211f0a-istiod-ca-cert\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.469585 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.469562 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-envoy\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.469974 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.469957 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7047851e-547c-4216-ac94-50c4bd211f0a-istio-podinfo\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.476576 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.476551 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-istio-token\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.477052 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.477035 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpvp2\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-kube-api-access-cpvp2\") pod \"e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.631866 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.631739 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:09.786843 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.786810 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz"] Jul 06 14:58:09.789920 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:58:09.789890 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7047851e_547c_4216_ac94_50c4bd211f0a.slice/crio-8e88e74b8a9daaeb5da9c6ec50ce3cbbf7b73771ff17e3763190a1c20d40525b WatchSource:0}: Error finding container 8e88e74b8a9daaeb5da9c6ec50ce3cbbf7b73771ff17e3763190a1c20d40525b: Status 404 returned error can't find the container with id 8e88e74b8a9daaeb5da9c6ec50ce3cbbf7b73771ff17e3763190a1c20d40525b Jul 06 14:58:09.792033 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.791991 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:09.792104 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.792057 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:09.792104 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:09.792087 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:10.381063 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:10.381020 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" event={"ID":"7047851e-547c-4216-ac94-50c4bd211f0a","Type":"ContainerStarted","Data":"d45fea64a0c7b80990dd961289c7c5e3fdf66400ef24a44671246a3eaa795b4e"} Jul 06 14:58:10.381507 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:10.381088 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" event={"ID":"7047851e-547c-4216-ac94-50c4bd211f0a","Type":"ContainerStarted","Data":"8e88e74b8a9daaeb5da9c6ec50ce3cbbf7b73771ff17e3763190a1c20d40525b"} Jul 06 14:58:10.408956 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:10.408882 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" podStartSLOduration=1.408859984 podStartE2EDuration="1.408859984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:58:10.407923634 +0000 UTC m=+2025.950611173" watchObservedRunningTime="2026-07-06 14:58:10.408859984 +0000 UTC m=+2025.951547523" Jul 06 14:58:10.632478 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:10.632376 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:10.637505 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:10.637478 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:11.386419 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:11.386385 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:11.387532 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:11.387509 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:12.562640 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.562599 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w"] Jul 06 14:58:12.563147 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.563125 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" Jul 06 14:58:12.563209 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.563150 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1648d32-c6ca-42c1-ac59-1ab0b50fa716" containerName="istio-proxy" Jul 06 14:58:12.566471 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.566446 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:12.569763 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.569718 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-3b4c5e-serving-cert\"" Jul 06 14:58:12.575514 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.575486 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w"] Jul 06 14:58:12.594119 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.594087 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe81d15d-8b58-471f-b397-9f32354e0340-openshift-service-ca\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:12.594311 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.594141 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:12.594311 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.594168 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6hpj\" (UniqueName: \"kubernetes.io/projected/fe81d15d-8b58-471f-b397-9f32354e0340-kube-api-access-d6hpj\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:12.695492 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.695459 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe81d15d-8b58-471f-b397-9f32354e0340-openshift-service-ca\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:12.695678 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.695512 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:12.695678 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.695532 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6hpj\" (UniqueName: \"kubernetes.io/projected/fe81d15d-8b58-471f-b397-9f32354e0340-kube-api-access-d6hpj\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:12.695678 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:12.695636 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-3b4c5e-serving-cert: secret "maas-api-e2e-api-a-3b4c5e-serving-cert" not found Jul 06 14:58:12.695808 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:12.695714 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls podName:fe81d15d-8b58-471f-b397-9f32354e0340 nodeName:}" failed. No retries permitted until 2026-07-06 14:58:13.19569472 +0000 UTC m=+2028.738382237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls") pod "maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" (UID: "fe81d15d-8b58-471f-b397-9f32354e0340") : secret "maas-api-e2e-api-a-3b4c5e-serving-cert" not found Jul 06 14:58:12.696237 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.696221 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe81d15d-8b58-471f-b397-9f32354e0340-openshift-service-ca\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:12.707090 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:12.707062 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6hpj\" (UniqueName: \"kubernetes.io/projected/fe81d15d-8b58-471f-b397-9f32354e0340-kube-api-access-d6hpj\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:13.201523 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:13.201481 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:13.204024 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:13.204002 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls\") pod \"maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:13.480203 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:13.480111 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:13.629566 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:13.629528 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w"] Jul 06 14:58:13.632723 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:58:13.632695 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe81d15d_8b58_471f_b397_9f32354e0340.slice/crio-e6439ed4bcb799b7a60c6f1863a786f6cdc6ebfd1e857aea185f2b0bbe584a4d WatchSource:0}: Error finding container e6439ed4bcb799b7a60c6f1863a786f6cdc6ebfd1e857aea185f2b0bbe584a4d: Status 404 returned error can't find the container with id e6439ed4bcb799b7a60c6f1863a786f6cdc6ebfd1e857aea185f2b0bbe584a4d Jul 06 14:58:14.402481 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:14.402443 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" event={"ID":"fe81d15d-8b58-471f-b397-9f32354e0340","Type":"ContainerStarted","Data":"e85d016243415d4ff31badec5fad700e668c431d79129e0b3d2569cc4b378a96"} Jul 06 14:58:14.402660 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:14.402505 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" event={"ID":"fe81d15d-8b58-471f-b397-9f32354e0340","Type":"ContainerStarted","Data":"e6439ed4bcb799b7a60c6f1863a786f6cdc6ebfd1e857aea185f2b0bbe584a4d"} Jul 06 14:58:14.402660 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:14.402578 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:14.430415 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:14.430312 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" podStartSLOduration=1.883715676 podStartE2EDuration="2.430290405s" totalImagesPullingTime="546.574729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:58:14.430081773 +0000 UTC m=+2029.972769311" watchObservedRunningTime="2026-07-06 14:58:14.430290405 +0000 UTC m=+2029.972977943" Jul 06 14:58:20.412968 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:20.412939 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:20.851561 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:20.851517 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v"] Jul 06 14:58:20.863811 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:20.863085 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:20.875813 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:20.869496 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-81b1b4-serving-cert\"" Jul 06 14:58:20.879811 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:20.877411 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v"] Jul 06 14:58:20.991486 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:20.991450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngttm\" (UniqueName: \"kubernetes.io/projected/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-kube-api-access-ngttm\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:20.991774 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:20.991752 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:20.991975 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:20.991959 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-openshift-service-ca\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.093188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.093152 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngttm\" (UniqueName: \"kubernetes.io/projected/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-kube-api-access-ngttm\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.093188 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.093201 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.093420 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.093232 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-openshift-service-ca\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.093420 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:21.093337 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-81b1b4-serving-cert: secret "maas-api-e2e-api-b-81b1b4-serving-cert" not found Jul 06 14:58:21.093420 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:21.093418 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls podName:7aeb7c68-dfac-43d3-bbbe-a448cbce36ed nodeName:}" failed. No retries permitted until 2026-07-06 14:58:21.593393311 +0000 UTC m=+2037.136080832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls") pod "maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" (UID: "7aeb7c68-dfac-43d3-bbbe-a448cbce36ed") : secret "maas-api-e2e-api-b-81b1b4-serving-cert" not found Jul 06 14:58:21.093900 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.093883 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-openshift-service-ca\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.106314 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.106239 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngttm\" (UniqueName: \"kubernetes.io/projected/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-kube-api-access-ngttm\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.599604 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.599568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.602147 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.602123 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls\") pod \"maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.795225 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.795189 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:21.981853 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:21.978417 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v"] Jul 06 14:58:21.986056 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:58:21.986013 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aeb7c68_dfac_43d3_bbbe_a448cbce36ed.slice/crio-f3d268d20ce80bdb430aad3f4924df1bfa934727d6646dcbb5d9cdf31f6b907b WatchSource:0}: Error finding container f3d268d20ce80bdb430aad3f4924df1bfa934727d6646dcbb5d9cdf31f6b907b: Status 404 returned error can't find the container with id f3d268d20ce80bdb430aad3f4924df1bfa934727d6646dcbb5d9cdf31f6b907b Jul 06 14:58:22.443289 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:22.443253 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" event={"ID":"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed","Type":"ContainerStarted","Data":"f3d268d20ce80bdb430aad3f4924df1bfa934727d6646dcbb5d9cdf31f6b907b"} Jul 06 14:58:23.449389 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:23.449355 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" event={"ID":"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed","Type":"ContainerStarted","Data":"1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7"} Jul 06 14:58:23.449772 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:23.449463 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:23.480116 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:23.480064 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" podStartSLOduration=2.972335373 podStartE2EDuration="3.480044138s" totalImagesPullingTime="507.708765ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:58:23.477993353 +0000 UTC m=+2039.020680892" watchObservedRunningTime="2026-07-06 14:58:23.480044138 +0000 UTC m=+2039.022731703" Jul 06 14:58:26.387988 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:26.387949 2565 patch_prober.go:28] interesting pod/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 06 14:58:26.388430 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:26.388020 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 06 14:58:26.467590 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:26.467559 2565 generic.go:356] "Generic (PLEG): container finished" podID="7047851e-547c-4216-ac94-50c4bd211f0a" containerID="d45fea64a0c7b80990dd961289c7c5e3fdf66400ef24a44671246a3eaa795b4e" exitCode=0 Jul 06 14:58:26.467766 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:26.467634 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" event={"ID":"7047851e-547c-4216-ac94-50c4bd211f0a","Type":"ContainerDied","Data":"d45fea64a0c7b80990dd961289c7c5e3fdf66400ef24a44671246a3eaa795b4e"} Jul 06 14:58:26.468112 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:26.468098 2565 scope.go:117] "RemoveContainer" containerID="d45fea64a0c7b80990dd961289c7c5e3fdf66400ef24a44671246a3eaa795b4e" Jul 06 14:58:26.469255 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:26.469226 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:26.469321 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:26.469310 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:26.469364 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:26.469348 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:27.478045 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:27.477998 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" event={"ID":"7047851e-547c-4216-ac94-50c4bd211f0a","Type":"ContainerStarted","Data":"70686de07a7358825e7764d2d67e7f3bea39d2d86023e3ee8871ddda6f871943"} Jul 06 14:58:27.632598 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:27.632556 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:27.632966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:27.632943 2565 patch_prober.go:28] interesting pod/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 06 14:58:27.633033 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:27.632997 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 06 14:58:28.632408 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:28.632372 2565 patch_prober.go:28] interesting pod/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 06 14:58:28.632865 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:28.632426 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 06 14:58:29.464152 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:29.464116 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:29.632703 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:29.632662 2565 patch_prober.go:28] interesting pod/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 06 14:58:29.633106 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:29.632714 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 06 14:58:30.633112 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:30.633073 2565 patch_prober.go:28] interesting pod/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 06 14:58:30.633555 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:30.633137 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 06 14:58:31.636387 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:31.636358 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:31.636977 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:31.636631 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:31.637323 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:31.637309 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:34.307044 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.307004 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w"] Jul 06 14:58:34.307517 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.307263 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" podUID="fe81d15d-8b58-471f-b397-9f32354e0340" containerName="maas-api" containerID="cri-o://e85d016243415d4ff31badec5fad700e668c431d79129e0b3d2569cc4b378a96" gracePeriod=30 Jul 06 14:58:34.518089 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.518054 2565 generic.go:356] "Generic (PLEG): container finished" podID="fe81d15d-8b58-471f-b397-9f32354e0340" containerID="e85d016243415d4ff31badec5fad700e668c431d79129e0b3d2569cc4b378a96" exitCode=0 Jul 06 14:58:34.518295 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.518140 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" event={"ID":"fe81d15d-8b58-471f-b397-9f32354e0340","Type":"ContainerDied","Data":"e85d016243415d4ff31badec5fad700e668c431d79129e0b3d2569cc4b378a96"} Jul 06 14:58:34.572688 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.572622 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:34.729469 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.729421 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls\") pod \"fe81d15d-8b58-471f-b397-9f32354e0340\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " Jul 06 14:58:34.729469 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.729486 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe81d15d-8b58-471f-b397-9f32354e0340-openshift-service-ca\") pod \"fe81d15d-8b58-471f-b397-9f32354e0340\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " Jul 06 14:58:34.729747 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.729522 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6hpj\" (UniqueName: \"kubernetes.io/projected/fe81d15d-8b58-471f-b397-9f32354e0340-kube-api-access-d6hpj\") pod \"fe81d15d-8b58-471f-b397-9f32354e0340\" (UID: \"fe81d15d-8b58-471f-b397-9f32354e0340\") " Jul 06 14:58:34.729914 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.729888 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe81d15d-8b58-471f-b397-9f32354e0340-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe81d15d-8b58-471f-b397-9f32354e0340" (UID: "fe81d15d-8b58-471f-b397-9f32354e0340"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:58:34.731840 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.731814 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe81d15d-8b58-471f-b397-9f32354e0340-kube-api-access-d6hpj" (OuterVolumeSpecName: "kube-api-access-d6hpj") pod "fe81d15d-8b58-471f-b397-9f32354e0340" (UID: "fe81d15d-8b58-471f-b397-9f32354e0340"). InnerVolumeSpecName "kube-api-access-d6hpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:58:34.731840 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.731823 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe81d15d-8b58-471f-b397-9f32354e0340" (UID: "fe81d15d-8b58-471f-b397-9f32354e0340"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:58:34.830621 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.830532 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe81d15d-8b58-471f-b397-9f32354e0340-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:34.830621 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.830564 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe81d15d-8b58-471f-b397-9f32354e0340-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:34.830621 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:34.830577 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6hpj\" (UniqueName: \"kubernetes.io/projected/fe81d15d-8b58-471f-b397-9f32354e0340-kube-api-access-d6hpj\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:35.524520 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:35.524485 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" event={"ID":"fe81d15d-8b58-471f-b397-9f32354e0340","Type":"ContainerDied","Data":"e6439ed4bcb799b7a60c6f1863a786f6cdc6ebfd1e857aea185f2b0bbe584a4d"} Jul 06 14:58:35.524520 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:35.524508 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w" Jul 06 14:58:35.525062 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:35.524559 2565 scope.go:117] "RemoveContainer" containerID="e85d016243415d4ff31badec5fad700e668c431d79129e0b3d2569cc4b378a96" Jul 06 14:58:35.554758 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:35.554724 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w"] Jul 06 14:58:35.554962 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:35.554861 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-3b4c5e-5f9f4c7c8d-4jk7w"] Jul 06 14:58:36.933269 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:36.933232 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe81d15d-8b58-471f-b397-9f32354e0340" path="/var/lib/kubelet/pods/fe81d15d-8b58-471f-b397-9f32354e0340/volumes" Jul 06 14:58:40.425484 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:40.425452 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz"] Jul 06 14:58:40.425914 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:40.425668 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" containerID="cri-o://70686de07a7358825e7764d2d67e7f3bea39d2d86023e3ee8871ddda6f871943" gracePeriod=30 Jul 06 14:58:41.849532 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:41.849496 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v"] Jul 06 14:58:41.850042 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:41.849818 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" podUID="7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" containerName="maas-api" containerID="cri-o://1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7" gracePeriod=30 Jul 06 14:58:42.113804 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.113719 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:42.201110 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.201077 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls\") pod \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " Jul 06 14:58:42.201305 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.201120 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-openshift-service-ca\") pod \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " Jul 06 14:58:42.201305 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.201161 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngttm\" (UniqueName: \"kubernetes.io/projected/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-kube-api-access-ngttm\") pod \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\" (UID: \"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed\") " Jul 06 14:58:42.201505 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.201478 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" (UID: "7aeb7c68-dfac-43d3-bbbe-a448cbce36ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:58:42.203457 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.203434 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" (UID: "7aeb7c68-dfac-43d3-bbbe-a448cbce36ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:58:42.203522 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.203435 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-kube-api-access-ngttm" (OuterVolumeSpecName: "kube-api-access-ngttm") pod "7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" (UID: "7aeb7c68-dfac-43d3-bbbe-a448cbce36ed"). InnerVolumeSpecName "kube-api-access-ngttm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:58:42.302593 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.302558 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngttm\" (UniqueName: \"kubernetes.io/projected/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-kube-api-access-ngttm\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:42.302593 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.302587 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:42.302593 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.302597 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:42.564641 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.564606 2565 generic.go:356] "Generic (PLEG): container finished" podID="7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" containerID="1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7" exitCode=0 Jul 06 14:58:42.564901 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.564662 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" event={"ID":"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed","Type":"ContainerDied","Data":"1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7"} Jul 06 14:58:42.564901 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.564685 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" Jul 06 14:58:42.564901 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.564731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v" event={"ID":"7aeb7c68-dfac-43d3-bbbe-a448cbce36ed","Type":"ContainerDied","Data":"f3d268d20ce80bdb430aad3f4924df1bfa934727d6646dcbb5d9cdf31f6b907b"} Jul 06 14:58:42.564901 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.564752 2565 scope.go:117] "RemoveContainer" containerID="1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7" Jul 06 14:58:42.574671 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.574649 2565 scope.go:117] "RemoveContainer" containerID="1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7" Jul 06 14:58:42.575014 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:42.574992 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7\": container with ID starting with 1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7 not found: ID does not exist" containerID="1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7" Jul 06 14:58:42.575076 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.575024 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7"} err="failed to get container status \"1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7\": rpc error: code = NotFound desc = could not find container \"1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7\": container with ID starting with 1b1ded27803712181b1c677584cb67dd24e92781cfa9731d01d70e0605b52fb7 not found: ID does not exist" Jul 06 14:58:42.596393 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.596356 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v"] Jul 06 14:58:42.601892 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.601866 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-81b1b4-7ddddd5b8c-9vc9v"] Jul 06 14:58:42.934538 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:42.934508 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" path="/var/lib/kubelet/pods/7aeb7c68-dfac-43d3-bbbe-a448cbce36ed/volumes" Jul 06 14:58:45.584680 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.584649 2565 generic.go:356] "Generic (PLEG): container finished" podID="7047851e-547c-4216-ac94-50c4bd211f0a" containerID="70686de07a7358825e7764d2d67e7f3bea39d2d86023e3ee8871ddda6f871943" exitCode=0 Jul 06 14:58:45.585086 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.584700 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" event={"ID":"7047851e-547c-4216-ac94-50c4bd211f0a","Type":"ContainerDied","Data":"70686de07a7358825e7764d2d67e7f3bea39d2d86023e3ee8871ddda6f871943"} Jul 06 14:58:45.585086 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.584753 2565 scope.go:117] "RemoveContainer" containerID="d45fea64a0c7b80990dd961289c7c5e3fdf66400ef24a44671246a3eaa795b4e" Jul 06 14:58:45.718419 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.718393 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:45.734835 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.734805 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-istio-token\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.734835 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.734842 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-certs\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.735100 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.734873 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7047851e-547c-4216-ac94-50c4bd211f0a-istio-podinfo\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.735100 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.734905 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-data\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.735100 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.734946 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-credential-socket\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.735100 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.734987 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-envoy\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.735100 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.735017 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7047851e-547c-4216-ac94-50c4bd211f0a-istiod-ca-cert\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.735100 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.735045 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-socket\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.735100 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.735092 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpvp2\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-kube-api-access-cpvp2\") pod \"7047851e-547c-4216-ac94-50c4bd211f0a\" (UID: \"7047851e-547c-4216-ac94-50c4bd211f0a\") " Jul 06 14:58:45.735446 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.735233 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:45.735446 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.735427 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-certs\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:45.735620 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.735570 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:45.735685 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.735636 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:45.735765 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.735748 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7047851e-547c-4216-ac94-50c4bd211f0a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:58:45.736624 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.736602 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-data" (OuterVolumeSpecName: "istio-data") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:45.737858 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.737834 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-istio-token" (OuterVolumeSpecName: "istio-token") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:58:45.738021 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.737990 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-kube-api-access-cpvp2" (OuterVolumeSpecName: "kube-api-access-cpvp2") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "kube-api-access-cpvp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:58:45.738133 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.738023 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7047851e-547c-4216-ac94-50c4bd211f0a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:58:45.738133 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.738030 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7047851e-547c-4216-ac94-50c4bd211f0a" (UID: "7047851e-547c-4216-ac94-50c4bd211f0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:58:45.836343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.836298 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-istio-token\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:45.836343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.836336 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7047851e-547c-4216-ac94-50c4bd211f0a-istio-podinfo\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:45.836343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.836346 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-data\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:45.836343 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.836355 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-credential-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:45.836622 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.836364 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-istio-envoy\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:45.836622 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.836372 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7047851e-547c-4216-ac94-50c4bd211f0a-istiod-ca-cert\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:45.836622 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.836382 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7047851e-547c-4216-ac94-50c4bd211f0a-workload-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:45.836622 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:45.836395 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpvp2\" (UniqueName: \"kubernetes.io/projected/7047851e-547c-4216-ac94-50c4bd211f0a-kube-api-access-cpvp2\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 14:58:46.590945 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:46.590913 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" Jul 06 14:58:46.591399 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:46.590912 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz" event={"ID":"7047851e-547c-4216-ac94-50c4bd211f0a","Type":"ContainerDied","Data":"8e88e74b8a9daaeb5da9c6ec50ce3cbbf7b73771ff17e3763190a1c20d40525b"} Jul 06 14:58:46.591399 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:46.591039 2565 scope.go:117] "RemoveContainer" containerID="70686de07a7358825e7764d2d67e7f3bea39d2d86023e3ee8871ddda6f871943" Jul 06 14:58:46.617490 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:46.617451 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz"] Jul 06 14:58:46.621479 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:46.621455 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3b4c5e-openshift-default-7db4cd4467-2c6cz"] Jul 06 14:58:46.933363 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:46.933275 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" path="/var/lib/kubelet/pods/7047851e-547c-4216-ac94-50c4bd211f0a/volumes" Jul 06 14:58:48.646209 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646174 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk"] Jul 06 14:58:48.646710 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646690 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" containerName="maas-api" Jul 06 14:58:48.646815 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646715 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" containerName="maas-api" Jul 06 14:58:48.646815 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646730 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe81d15d-8b58-471f-b397-9f32354e0340" containerName="maas-api" Jul 06 14:58:48.646815 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646739 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe81d15d-8b58-471f-b397-9f32354e0340" containerName="maas-api" Jul 06 14:58:48.646815 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646772 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" Jul 06 14:58:48.646815 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646802 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" Jul 06 14:58:48.646815 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646812 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" Jul 06 14:58:48.647125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646821 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" Jul 06 14:58:48.647125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646919 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe81d15d-8b58-471f-b397-9f32354e0340" containerName="maas-api" Jul 06 14:58:48.647125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646935 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" Jul 06 14:58:48.647125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646946 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7047851e-547c-4216-ac94-50c4bd211f0a" containerName="istio-proxy" Jul 06 14:58:48.647125 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.646957 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aeb7c68-dfac-43d3-bbbe-a448cbce36ed" containerName="maas-api" Jul 06 14:58:48.651198 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.651173 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.653936 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.653913 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-e63e88-openshift-default-dockercfg-8khqb\"" Jul 06 14:58:48.661697 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.661669 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk"] Jul 06 14:58:48.763124 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763072 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-podinfo\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.763298 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763142 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-certs\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.763298 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763177 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-token\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.763432 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763288 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-socket\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.763432 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763327 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-credential-socket\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.763432 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763347 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jztb\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-kube-api-access-9jztb\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.763432 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763369 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-envoy\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.763432 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763389 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ed1d93-21ce-490d-a100-a6f523945dcb-istiod-ca-cert\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.763682 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.763473 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-data\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864047 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.863991 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-socket\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864047 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-credential-socket\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864304 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864085 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jztb\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-kube-api-access-9jztb\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864304 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864116 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-envoy\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864304 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864148 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ed1d93-21ce-490d-a100-a6f523945dcb-istiod-ca-cert\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864304 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864206 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-data\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864525 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-podinfo\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864525 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864358 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-certs\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864525 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864383 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-token\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864525 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864476 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-socket\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864742 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864575 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-credential-socket\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.864903 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864876 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-data\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.865044 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.864926 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-certs\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.865044 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.865002 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ed1d93-21ce-490d-a100-a6f523945dcb-istiod-ca-cert\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.866966 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.866945 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-podinfo\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.867070 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.866981 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-envoy\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.873018 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.872988 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jztb\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-kube-api-access-9jztb\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.873340 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.873317 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-token\") pod \"e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:48.963442 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:48.963393 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:49.114678 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:49.114643 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk"] Jul 06 14:58:49.118410 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:58:49.118381 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88ed1d93_21ce_490d_a100_a6f523945dcb.slice/crio-e57fbbfb2dc3e01a6f507ee7744f372b13dd3c3a35fe0fd6251600d612c9efcd WatchSource:0}: Error finding container e57fbbfb2dc3e01a6f507ee7744f372b13dd3c3a35fe0fd6251600d612c9efcd: Status 404 returned error can't find the container with id e57fbbfb2dc3e01a6f507ee7744f372b13dd3c3a35fe0fd6251600d612c9efcd Jul 06 14:58:49.121042 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:49.121006 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:49.121162 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:49.121095 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:49.121162 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:49.121145 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:58:49.607366 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:49.607332 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" event={"ID":"88ed1d93-21ce-490d-a100-a6f523945dcb","Type":"ContainerStarted","Data":"774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7"} Jul 06 14:58:49.607583 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:49.607385 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" event={"ID":"88ed1d93-21ce-490d-a100-a6f523945dcb","Type":"ContainerStarted","Data":"e57fbbfb2dc3e01a6f507ee7744f372b13dd3c3a35fe0fd6251600d612c9efcd"} Jul 06 14:58:49.631663 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:49.631605 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" podStartSLOduration=1.6315867339999999 podStartE2EDuration="1.631586734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:58:49.628634857 +0000 UTC m=+2065.171322394" watchObservedRunningTime="2026-07-06 14:58:49.631586734 +0000 UTC m=+2065.174274271" Jul 06 14:58:49.963983 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:49.963935 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:50.968878 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:50.968849 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:51.616240 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:51.616208 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:51.617350 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:51.617330 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:58:51.938107 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:51.938071 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf"] Jul 06 14:58:51.941800 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:51.941766 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:51.946265 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:51.946241 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-e63e88-serving-cert\"" Jul 06 14:58:51.957253 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:51.957223 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf"] Jul 06 14:58:52.000811 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.000742 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87a86563-ab5b-418a-8ca2-9619119200c2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.001208 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.000837 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlplv\" (UniqueName: \"kubernetes.io/projected/87a86563-ab5b-418a-8ca2-9619119200c2-kube-api-access-nlplv\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.001208 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.000904 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.101730 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.101686 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlplv\" (UniqueName: \"kubernetes.io/projected/87a86563-ab5b-418a-8ca2-9619119200c2-kube-api-access-nlplv\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.101972 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.101754 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.101972 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.101846 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87a86563-ab5b-418a-8ca2-9619119200c2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.101972 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:52.101913 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-e63e88-serving-cert: secret "maas-api-e2e-shared-a-e63e88-serving-cert" not found Jul 06 14:58:52.102153 ip-10-0-128-205 kubenswrapper[2565]: E0706 14:58:52.101995 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls podName:87a86563-ab5b-418a-8ca2-9619119200c2 nodeName:}" failed. No retries permitted until 2026-07-06 14:58:52.601973368 +0000 UTC m=+2068.144660886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls") pod "maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" (UID: "87a86563-ab5b-418a-8ca2-9619119200c2") : secret "maas-api-e2e-shared-a-e63e88-serving-cert" not found Jul 06 14:58:52.102691 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.102672 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87a86563-ab5b-418a-8ca2-9619119200c2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.116726 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.116689 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlplv\" (UniqueName: \"kubernetes.io/projected/87a86563-ab5b-418a-8ca2-9619119200c2-kube-api-access-nlplv\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.606628 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.606587 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.609134 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.609114 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls\") pod \"maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:52.854735 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:52.854695 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:53.008279 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:53.008252 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf"] Jul 06 14:58:53.010121 ip-10-0-128-205 kubenswrapper[2565]: W0706 14:58:53.010095 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a86563_ab5b_418a_8ca2_9619119200c2.slice/crio-91240ecf67d7844da90e7f7c1f89f5141da878247a963eb19116bea71b0183da WatchSource:0}: Error finding container 91240ecf67d7844da90e7f7c1f89f5141da878247a963eb19116bea71b0183da: Status 404 returned error can't find the container with id 91240ecf67d7844da90e7f7c1f89f5141da878247a963eb19116bea71b0183da Jul 06 14:58:53.630166 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:53.630050 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" event={"ID":"87a86563-ab5b-418a-8ca2-9619119200c2","Type":"ContainerStarted","Data":"30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f"} Jul 06 14:58:53.630166 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:53.630112 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" event={"ID":"87a86563-ab5b-418a-8ca2-9619119200c2","Type":"ContainerStarted","Data":"91240ecf67d7844da90e7f7c1f89f5141da878247a963eb19116bea71b0183da"} Jul 06 14:58:53.630401 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:53.630328 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:58:53.667800 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:53.667725 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" podStartSLOduration=2.310373731 podStartE2EDuration="2.667708546s" totalImagesPullingTime="357.334815ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:58:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:58:53.666575046 +0000 UTC m=+2069.209262596" watchObservedRunningTime="2026-07-06 14:58:53.667708546 +0000 UTC m=+2069.210396083" Jul 06 14:58:59.640547 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:58:59.640515 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 14:59:06.617849 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:06.617811 2565 patch_prober.go:28] interesting pod/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 06 14:59:06.618296 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:06.617879 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 06 14:59:06.704906 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:06.704870 2565 generic.go:356] "Generic (PLEG): container finished" podID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerID="774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7" exitCode=0 Jul 06 14:59:06.705077 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:06.704924 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" event={"ID":"88ed1d93-21ce-490d-a100-a6f523945dcb","Type":"ContainerDied","Data":"774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7"} Jul 06 14:59:06.705338 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:06.705322 2565 scope.go:117] "RemoveContainer" containerID="774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7" Jul 06 14:59:06.706367 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:06.706339 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:59:06.706425 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:06.706409 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:59:06.706462 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:06.706443 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 14:59:07.711446 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:07.711413 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" event={"ID":"88ed1d93-21ce-490d-a100-a6f523945dcb","Type":"ContainerStarted","Data":"170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583"} Jul 06 14:59:07.964526 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:07.964420 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:59:07.964717 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:07.964561 2565 patch_prober.go:28] interesting pod/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 06 14:59:07.964717 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:07.964619 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 06 14:59:08.964260 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:08.964211 2565 patch_prober.go:28] interesting pod/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 06 14:59:08.964807 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:08.964284 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 06 14:59:09.964362 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:09.964318 2565 patch_prober.go:28] interesting pod/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 06 14:59:09.965054 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:09.964950 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 06 14:59:10.963962 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:10.963913 2565 patch_prober.go:28] interesting pod/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 06 14:59:10.964196 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:10.964005 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 06 14:59:11.968025 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:11.967990 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:59:11.968498 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:11.968368 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 14:59:11.968993 ip-10-0-128-205 kubenswrapper[2565]: I0706 14:59:11.968973 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 15:04:22.567187 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:22.567144 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf"] Jul 06 15:04:22.568766 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:22.567403 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" podUID="87a86563-ab5b-418a-8ca2-9619119200c2" containerName="maas-api" containerID="cri-o://30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f" gracePeriod=30 Jul 06 15:04:23.025426 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.025400 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 15:04:23.136026 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.135924 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls\") pod \"87a86563-ab5b-418a-8ca2-9619119200c2\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " Jul 06 15:04:23.136210 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.136050 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87a86563-ab5b-418a-8ca2-9619119200c2-openshift-service-ca\") pod \"87a86563-ab5b-418a-8ca2-9619119200c2\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " Jul 06 15:04:23.136210 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.136076 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlplv\" (UniqueName: \"kubernetes.io/projected/87a86563-ab5b-418a-8ca2-9619119200c2-kube-api-access-nlplv\") pod \"87a86563-ab5b-418a-8ca2-9619119200c2\" (UID: \"87a86563-ab5b-418a-8ca2-9619119200c2\") " Jul 06 15:04:23.136461 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.136433 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87a86563-ab5b-418a-8ca2-9619119200c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87a86563-ab5b-418a-8ca2-9619119200c2" (UID: "87a86563-ab5b-418a-8ca2-9619119200c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:04:23.138151 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.138125 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87a86563-ab5b-418a-8ca2-9619119200c2" (UID: "87a86563-ab5b-418a-8ca2-9619119200c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:04:23.138244 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.138223 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87a86563-ab5b-418a-8ca2-9619119200c2-kube-api-access-nlplv" (OuterVolumeSpecName: "kube-api-access-nlplv") pod "87a86563-ab5b-418a-8ca2-9619119200c2" (UID: "87a86563-ab5b-418a-8ca2-9619119200c2"). InnerVolumeSpecName "kube-api-access-nlplv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:04:23.237073 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.237039 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87a86563-ab5b-418a-8ca2-9619119200c2-openshift-service-ca\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:04:23.237073 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.237062 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlplv\" (UniqueName: \"kubernetes.io/projected/87a86563-ab5b-418a-8ca2-9619119200c2-kube-api-access-nlplv\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:04:23.237073 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.237073 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87a86563-ab5b-418a-8ca2-9619119200c2-maas-api-tls\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:04:23.239670 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.239640 2565 generic.go:356] "Generic (PLEG): container finished" podID="87a86563-ab5b-418a-8ca2-9619119200c2" containerID="30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f" exitCode=0 Jul 06 15:04:23.239845 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.239685 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" Jul 06 15:04:23.239845 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.239701 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" event={"ID":"87a86563-ab5b-418a-8ca2-9619119200c2","Type":"ContainerDied","Data":"30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f"} Jul 06 15:04:23.239845 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.239757 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf" event={"ID":"87a86563-ab5b-418a-8ca2-9619119200c2","Type":"ContainerDied","Data":"91240ecf67d7844da90e7f7c1f89f5141da878247a963eb19116bea71b0183da"} Jul 06 15:04:23.239845 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.239815 2565 scope.go:117] "RemoveContainer" containerID="30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f" Jul 06 15:04:23.249207 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.249184 2565 scope.go:117] "RemoveContainer" containerID="30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f" Jul 06 15:04:23.249451 ip-10-0-128-205 kubenswrapper[2565]: E0706 15:04:23.249432 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f\": container with ID starting with 30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f not found: ID does not exist" containerID="30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f" Jul 06 15:04:23.249528 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.249464 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f"} err="failed to get container status \"30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f\": rpc error: code = NotFound desc = could not find container \"30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f\": container with ID starting with 30e72a758aaef88a5283af9d76b7edd103b8b0f6ea37dfb33cb04d191d4ee86f not found: ID does not exist" Jul 06 15:04:23.262578 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.262546 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf"] Jul 06 15:04:23.270763 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:23.270737 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e63e88-84679797f6-9pfcf"] Jul 06 15:04:24.933984 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:04:24.933957 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87a86563-ab5b-418a-8ca2-9619119200c2" path="/var/lib/kubelet/pods/87a86563-ab5b-418a-8ca2-9619119200c2/volumes" Jul 06 15:05:04.142323 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:04.142290 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk"] Jul 06 15:05:04.142895 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:04.142552 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" containerID="cri-o://170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583" gracePeriod=30 Jul 06 15:05:09.401742 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.401715 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 15:05:09.461147 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.461109 2565 generic.go:356] "Generic (PLEG): container finished" podID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerID="170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583" exitCode=0 Jul 06 15:05:09.461352 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.461168 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" Jul 06 15:05:09.461352 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.461191 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" event={"ID":"88ed1d93-21ce-490d-a100-a6f523945dcb","Type":"ContainerDied","Data":"170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583"} Jul 06 15:05:09.461352 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.461254 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk" event={"ID":"88ed1d93-21ce-490d-a100-a6f523945dcb","Type":"ContainerDied","Data":"e57fbbfb2dc3e01a6f507ee7744f372b13dd3c3a35fe0fd6251600d612c9efcd"} Jul 06 15:05:09.461352 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.461283 2565 scope.go:117] "RemoveContainer" containerID="170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583" Jul 06 15:05:09.462474 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462455 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-certs\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.462536 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462510 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ed1d93-21ce-490d-a100-a6f523945dcb-istiod-ca-cert\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.462721 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462564 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-podinfo\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.462721 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462596 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jztb\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-kube-api-access-9jztb\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.462721 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462639 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-socket\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.462721 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462689 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-credential-socket\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.462721 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462717 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-data\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.463002 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462743 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-token\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.463002 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.462763 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-envoy\") pod \"88ed1d93-21ce-490d-a100-a6f523945dcb\" (UID: \"88ed1d93-21ce-490d-a100-a6f523945dcb\") " Jul 06 15:05:09.463109 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.463053 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88ed1d93-21ce-490d-a100-a6f523945dcb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:05:09.463109 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.463053 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:05:09.463322 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.463273 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-credential-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.463322 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.463296 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ed1d93-21ce-490d-a100-a6f523945dcb-istiod-ca-cert\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.463322 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.463303 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:05:09.463554 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.463452 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:05:09.464395 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.464354 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-data" (OuterVolumeSpecName: "istio-data") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:05:09.465446 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.465405 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-kube-api-access-9jztb" (OuterVolumeSpecName: "kube-api-access-9jztb") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "kube-api-access-9jztb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:05:09.465561 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.465538 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 15:05:09.465820 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.465767 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:05:09.465820 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.465778 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-token" (OuterVolumeSpecName: "istio-token") pod "88ed1d93-21ce-490d-a100-a6f523945dcb" (UID: "88ed1d93-21ce-490d-a100-a6f523945dcb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:05:09.486939 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.486901 2565 scope.go:117] "RemoveContainer" containerID="774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7" Jul 06 15:05:09.495560 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.495537 2565 scope.go:117] "RemoveContainer" containerID="170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583" Jul 06 15:05:09.495881 ip-10-0-128-205 kubenswrapper[2565]: E0706 15:05:09.495853 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583\": container with ID starting with 170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583 not found: ID does not exist" containerID="170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583" Jul 06 15:05:09.495952 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.495896 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583"} err="failed to get container status \"170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583\": rpc error: code = NotFound desc = could not find container \"170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583\": container with ID starting with 170924a0ae2c14ac2a5befbfaa45f294ba87c8271503f58597b4be4767180583 not found: ID does not exist" Jul 06 15:05:09.495952 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.495916 2565 scope.go:117] "RemoveContainer" containerID="774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7" Jul 06 15:05:09.496149 ip-10-0-128-205 kubenswrapper[2565]: E0706 15:05:09.496131 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7\": container with ID starting with 774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7 not found: ID does not exist" containerID="774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7" Jul 06 15:05:09.496206 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.496157 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7"} err="failed to get container status \"774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7\": rpc error: code = NotFound desc = could not find container \"774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7\": container with ID starting with 774bf5264b1245c205b463b7074df2eaaad35a322954cfcafcbdd4feeac24ea7 not found: ID does not exist" Jul 06 15:05:09.564761 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.564718 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-podinfo\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.564761 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.564749 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jztb\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-kube-api-access-9jztb\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.564761 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.564761 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-socket\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.564761 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.564769 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-data\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.565068 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.564778 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-token\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.565068 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.564803 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-istio-envoy\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.565068 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.564814 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ed1d93-21ce-490d-a100-a6f523945dcb-workload-certs\") on node \"ip-10-0-128-205.ec2.internal\" DevicePath \"\"" Jul 06 15:05:09.785306 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.785270 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk"] Jul 06 15:05:09.789492 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:09.789459 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-e63e88-openshift-default-7c84696895-5ftfk"] Jul 06 15:05:10.933144 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:10.933108 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" path="/var/lib/kubelet/pods/88ed1d93-21ce-490d-a100-a6f523945dcb/volumes" Jul 06 15:05:17.255687 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:17.255653 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6f8d48684c-kjdd9_88eafa99-3b3c-42bc-b23b-9dd6093ae909/maas-api/0.log" Jul 06 15:05:18.303008 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:18.302975 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9447c645c-nxt5w_3a1d148e-781a-4253-a4f7-b3b6805208f0/manager/0.log" Jul 06 15:05:18.310856 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:18.310835 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9447c645c-nxt5w_3a1d148e-781a-4253-a4f7-b3b6805208f0/copy-manifests/0.log" Jul 06 15:05:18.434736 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:18.434704 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-rrwm4_8ad1a72c-32ba-4e87-a08a-2809561453a4/postgres/0.log" Jul 06 15:05:20.064418 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:20.064377 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2cnhp_5e5d2fe0-7cff-4282-8b00-5d482db88099/manager/0.log" Jul 06 15:05:20.177375 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:20.177343 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-84q7v_32b889b1-0ea1-4cf9-a902-31e64a7f8e95/manager/0.log" Jul 06 15:05:20.290059 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:20.290031 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gcprq_321780b5-a07c-47f1-8fcb-913731118317/kuadrant-console-plugin/0.log" Jul 06 15:05:20.413682 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:20.413603 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jdb9x_a1d0f677-028c-4de7-ad27-8c7dc3496ed2/registry-server/0.log" Jul 06 15:05:20.549426 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:20.549398 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-x5jsd_51d430df-1114-4aea-a6b9-74f405c0359c/manager/0.log" Jul 06 15:05:20.814695 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:20.814662 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gfr62_afe2556f-be20-4740-9ccc-7894305d6442/manager/0.log" Jul 06 15:05:21.320807 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:21.320760 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fxrs2_8b23543c-75ef-4b27-8e6f-40a470c76d90/discovery/0.log" Jul 06 15:05:21.687910 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:21.687876 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pnkwz_dcfe503a-8674-4fdb-9e90-0d13732db389/istio-proxy/1.log" Jul 06 15:05:21.805645 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:21.805606 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tbvng_2b9f8f57-3a1a-467a-a6d3-0fab2de54f82/payload-pre-processing/0.log" Jul 06 15:05:21.936457 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:21.936407 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-pfcjl_17e0d943-5abe-4fe1-a442-5b27b0776b65/payload-processing/0.log" Jul 06 15:05:22.051043 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:22.050965 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5477b4dcdf-nnrp4_5e221b5b-b395-4184-9dfd-709037b5d5fd/router/0.log" Jul 06 15:05:22.526536 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:22.526505 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6s2c8_3640fc11-257a-453b-888d-83443aab1f53/main/0.log" Jul 06 15:05:22.764759 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:22.764730 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblspq8_8b589bea-9a59-4b61-a999-ce459c8f24ad/main/0.log" Jul 06 15:05:22.882806 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:22.882695 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lg87l_d0cadaf1-851d-42ec-b619-a15d120fc826/main/0.log" Jul 06 15:05:29.800770 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:29.800731 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rjl52_fcf896e5-cbff-43fa-8d73-b05d9be1a499/global-pull-secret-syncer/0.log" Jul 06 15:05:29.988905 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:29.988857 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-f9s5h_65638096-89bf-4730-b736-4412ccfd6c62/konnectivity-agent/0.log" Jul 06 15:05:30.011751 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:30.011722 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-205.ec2.internal_fbc6eb3049ab39b232b7d93c6478c5b8/haproxy/0.log" Jul 06 15:05:35.751573 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:35.751530 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2cnhp_5e5d2fe0-7cff-4282-8b00-5d482db88099/manager/0.log" Jul 06 15:05:35.776665 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:35.776632 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-84q7v_32b889b1-0ea1-4cf9-a902-31e64a7f8e95/manager/0.log" Jul 06 15:05:35.804444 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:35.804413 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gcprq_321780b5-a07c-47f1-8fcb-913731118317/kuadrant-console-plugin/0.log" Jul 06 15:05:35.857216 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:35.857184 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jdb9x_a1d0f677-028c-4de7-ad27-8c7dc3496ed2/registry-server/0.log" Jul 06 15:05:36.061189 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:36.061094 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-x5jsd_51d430df-1114-4aea-a6b9-74f405c0359c/manager/0.log" Jul 06 15:05:36.180400 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:36.180362 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gfr62_afe2556f-be20-4740-9ccc-7894305d6442/manager/0.log" Jul 06 15:05:38.395330 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:38.395293 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-64lkd_d667376e-1eea-44d0-b0ef-ce35edc8e783/node-exporter/0.log" Jul 06 15:05:38.421250 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:38.421214 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-64lkd_d667376e-1eea-44d0-b0ef-ce35edc8e783/kube-rbac-proxy/0.log" Jul 06 15:05:38.446022 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:38.445997 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-64lkd_d667376e-1eea-44d0-b0ef-ce35edc8e783/init-textfile/0.log" Jul 06 15:05:39.313009 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.312979 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m"] Jul 06 15:05:39.313476 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.313458 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87a86563-ab5b-418a-8ca2-9619119200c2" containerName="maas-api" Jul 06 15:05:39.313476 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.313477 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87a86563-ab5b-418a-8ca2-9619119200c2" containerName="maas-api" Jul 06 15:05:39.313617 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.313520 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" Jul 06 15:05:39.313617 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.313526 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" Jul 06 15:05:39.313617 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.313535 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" Jul 06 15:05:39.313617 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.313540 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" Jul 06 15:05:39.313617 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.313595 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87a86563-ab5b-418a-8ca2-9619119200c2" containerName="maas-api" Jul 06 15:05:39.313617 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.313604 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="88ed1d93-21ce-490d-a100-a6f523945dcb" containerName="istio-proxy" Jul 06 15:05:39.316504 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.316489 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.319417 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.319396 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8frwf\"/\"kube-root-ca.crt\"" Jul 06 15:05:39.320546 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.320528 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8frwf\"/\"openshift-service-ca.crt\"" Jul 06 15:05:39.320635 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.320556 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8frwf\"/\"default-dockercfg-dstt7\"" Jul 06 15:05:39.325687 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.325664 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m"] Jul 06 15:05:39.427930 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.427889 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qrx6\" (UniqueName: \"kubernetes.io/projected/03aabddd-6e8d-4c14-93c5-9a670b98d935-kube-api-access-5qrx6\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.428367 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.428010 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-proc\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.428367 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.428079 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-sys\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.428367 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.428109 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-podres\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.428367 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.428162 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-lib-modules\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.528850 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.528809 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-proc\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.529046 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.528873 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-sys\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.529046 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.528893 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-podres\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.529046 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.528911 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-lib-modules\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.529046 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.528935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qrx6\" (UniqueName: \"kubernetes.io/projected/03aabddd-6e8d-4c14-93c5-9a670b98d935-kube-api-access-5qrx6\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.529046 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.528947 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-proc\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.529046 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.529037 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-sys\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.529286 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.529058 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-lib-modules\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.529286 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.529072 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/03aabddd-6e8d-4c14-93c5-9a670b98d935-podres\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.537777 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.537753 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qrx6\" (UniqueName: \"kubernetes.io/projected/03aabddd-6e8d-4c14-93c5-9a670b98d935-kube-api-access-5qrx6\") pod \"perf-node-gather-daemonset-mw94m\" (UID: \"03aabddd-6e8d-4c14-93c5-9a670b98d935\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.626837 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.626714 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:39.765921 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:39.765890 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m"] Jul 06 15:05:39.768201 ip-10-0-128-205 kubenswrapper[2565]: W0706 15:05:39.768167 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod03aabddd_6e8d_4c14_93c5_9a670b98d935.slice/crio-f28fa9e4892c16e037e7cbaae52b2f6ac6dc3f1a8b734d9ac78ebd09a6ea7f52 WatchSource:0}: Error finding container f28fa9e4892c16e037e7cbaae52b2f6ac6dc3f1a8b734d9ac78ebd09a6ea7f52: Status 404 returned error can't find the container with id f28fa9e4892c16e037e7cbaae52b2f6ac6dc3f1a8b734d9ac78ebd09a6ea7f52 Jul 06 15:05:40.610329 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:40.610292 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" event={"ID":"03aabddd-6e8d-4c14-93c5-9a670b98d935","Type":"ContainerStarted","Data":"0368b9323ef6f0bf5e48cf35dcf00c2a6d98d26079dc9d63b106db00bed1198d"} Jul 06 15:05:40.610774 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:40.610345 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" event={"ID":"03aabddd-6e8d-4c14-93c5-9a670b98d935","Type":"ContainerStarted","Data":"f28fa9e4892c16e037e7cbaae52b2f6ac6dc3f1a8b734d9ac78ebd09a6ea7f52"} Jul 06 15:05:40.610774 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:40.610397 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:40.630711 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:40.630657 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" podStartSLOduration=1.63063974 podStartE2EDuration="1.63063974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:05:40.630166033 +0000 UTC m=+2476.172853581" watchObservedRunningTime="2026-07-06 15:05:40.63063974 +0000 UTC m=+2476.173327277" Jul 06 15:05:42.209923 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:42.209892 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-2lqbr_75ec02f5-9d55-4fea-b1ab-f8ec18d7151a/volume-data-source-validator/0.log" Jul 06 15:05:43.312597 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:43.312566 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8psvj_a04e7fb5-c4dc-48be-9748-428d658f9ca1/dns/0.log" Jul 06 15:05:43.336360 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:43.336325 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8psvj_a04e7fb5-c4dc-48be-9748-428d658f9ca1/kube-rbac-proxy/0.log" Jul 06 15:05:43.485514 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:43.485489 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5rtnh_3e3a655e-a83a-43a2-b610-6b66bf7c6fc0/dns-node-resolver/0.log" Jul 06 15:05:44.142822 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:44.142763 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-d75689b57-26vdj_4b873ff1-9a75-4ee5-9986-c3adca081c49/registry/0.log" Jul 06 15:05:44.197804 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:44.197762 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kfnjw_8ca6503a-77c2-47ef-8587-531b6ee00e34/node-ca/0.log" Jul 06 15:05:45.407017 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:45.406990 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fxrs2_8b23543c-75ef-4b27-8e6f-40a470c76d90/discovery/0.log" Jul 06 15:05:45.490126 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:45.490092 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pnkwz_dcfe503a-8674-4fdb-9e90-0d13732db389/istio-proxy/0.log" Jul 06 15:05:45.546762 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:45.546725 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pnkwz_dcfe503a-8674-4fdb-9e90-0d13732db389/istio-proxy/1.log" Jul 06 15:05:45.587358 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:45.587329 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tbvng_2b9f8f57-3a1a-467a-a6d3-0fab2de54f82/payload-pre-processing/0.log" Jul 06 15:05:45.669564 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:45.669466 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-pfcjl_17e0d943-5abe-4fe1-a442-5b27b0776b65/payload-processing/0.log" Jul 06 15:05:45.694633 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:45.694601 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5477b4dcdf-nnrp4_5e221b5b-b395-4184-9dfd-709037b5d5fd/router/0.log" Jul 06 15:05:46.451293 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:46.451250 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fr7mf_abb581bd-675d-43c4-9e11-da3095e4109d/serve-healthcheck-canary/0.log" Jul 06 15:05:46.626135 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:46.626104 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-mw94m" Jul 06 15:05:47.123390 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:47.123366 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-whrlv_8f0a0a51-50a8-4499-b7dd-c2445bb41b8f/kube-rbac-proxy/0.log" Jul 06 15:05:47.156245 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:47.156212 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-whrlv_8f0a0a51-50a8-4499-b7dd-c2445bb41b8f/exporter/0.log" Jul 06 15:05:47.183922 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:47.183869 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-whrlv_8f0a0a51-50a8-4499-b7dd-c2445bb41b8f/extractor/0.log" Jul 06 15:05:50.262900 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.262871 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6f8d48684c-kjdd9_88eafa99-3b3c-42bc-b23b-9dd6093ae909/maas-api/0.log" Jul 06 15:05:50.402758 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.402722 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fxrs2_8b23543c-75ef-4b27-8e6f-40a470c76d90/discovery/0.log" Jul 06 15:05:50.464980 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.464891 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pnkwz_dcfe503a-8674-4fdb-9e90-0d13732db389/istio-proxy/0.log" Jul 06 15:05:50.522038 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.521989 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pnkwz_dcfe503a-8674-4fdb-9e90-0d13732db389/istio-proxy/1.log" Jul 06 15:05:50.554278 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.554242 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tbvng_2b9f8f57-3a1a-467a-a6d3-0fab2de54f82/payload-pre-processing/0.log" Jul 06 15:05:50.623849 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.623820 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-pfcjl_17e0d943-5abe-4fe1-a442-5b27b0776b65/payload-processing/0.log" Jul 06 15:05:50.640083 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.640054 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5477b4dcdf-nnrp4_5e221b5b-b395-4184-9dfd-709037b5d5fd/router/0.log" Jul 06 15:05:50.823585 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.823495 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9447c645c-nxt5w_3a1d148e-781a-4253-a4f7-b3b6805208f0/manager/0.log" Jul 06 15:05:50.847343 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.847315 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9447c645c-nxt5w_3a1d148e-781a-4253-a4f7-b3b6805208f0/copy-manifests/0.log" Jul 06 15:05:50.874418 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:50.874391 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-rrwm4_8ad1a72c-32ba-4e87-a08a-2809561453a4/postgres/0.log" Jul 06 15:05:51.987286 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:51.987255 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fxrs2_8b23543c-75ef-4b27-8e6f-40a470c76d90/discovery/0.log" Jul 06 15:05:52.046514 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:52.046487 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pnkwz_dcfe503a-8674-4fdb-9e90-0d13732db389/istio-proxy/0.log" Jul 06 15:05:52.108057 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:52.108022 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pnkwz_dcfe503a-8674-4fdb-9e90-0d13732db389/istio-proxy/1.log" Jul 06 15:05:52.137773 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:52.137739 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tbvng_2b9f8f57-3a1a-467a-a6d3-0fab2de54f82/payload-pre-processing/0.log" Jul 06 15:05:52.203038 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:52.202951 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-pfcjl_17e0d943-5abe-4fe1-a442-5b27b0776b65/payload-processing/0.log" Jul 06 15:05:52.233063 ip-10-0-128-205 kubenswrapper[2565]: I0706 15:05:52.233030 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5477b4dcdf-nnrp4_5e221b5b-b395-4184-9dfd-709037b5d5fd/router/0.log"