Aug 04 12:25:31.196517 ip-10-0-132-96 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Aug 04 12:25:31.196527 ip-10-0-132-96 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Aug 04 12:25:31.196536 ip-10-0-132-96 systemd[1]: kubelet.service: Failed with result 'resources'. Aug 04 12:25:31.196852 ip-10-0-132-96 systemd[1]: Failed to start Kubernetes Kubelet. Aug 04 12:25:42.704282 ip-10-0-132-96 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Aug 04 12:25:42.704301 ip-10-0-132-96 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 47bf3bef41ec4f2788c09703f7ead9e4 -- Aug 04 12:27:19.635173 ip-10-0-132-96 systemd[1]: Starting Kubernetes Kubelet... Aug 04 12:27:20.075959 ip-10-0-132-96 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 12:27:20.075959 ip-10-0-132-96 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 12:27:20.075959 ip-10-0-132-96 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 12:27:20.075959 ip-10-0-132-96 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 12:27:20.075959 ip-10-0-132-96 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 12:27:20.079594 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.079473 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 12:27:20.085317 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085299 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 12:27:20.085317 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085316 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 12:27:20.085317 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085319 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085322 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085325 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085328 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085332 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085334 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085337 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085340 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085343 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085345 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085348 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085351 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085356 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085359 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085363 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085366 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085369 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085372 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085375 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 12:27:20.085409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085378 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085380 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085383 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085386 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085388 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085391 2580 feature_gate.go:328] unrecognized feature gate: Example Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085394 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085397 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085399 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085402 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085404 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085408 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085412 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085417 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085420 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085423 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085426 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085428 2580 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085431 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085433 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 12:27:20.085890 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085436 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085438 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085441 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085443 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085446 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085449 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085452 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085454 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085457 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085459 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085462 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085464 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085466 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085469 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085472 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085474 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085477 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085479 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085482 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085485 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 12:27:20.086377 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085487 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085489 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085492 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085495 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085497 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085500 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085503 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085505 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085508 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085512 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085515 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085518 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085521 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085524 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085526 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085528 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085531 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085533 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085536 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085539 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 12:27:20.086882 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085541 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085543 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085546 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085548 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085551 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085554 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085556 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085559 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085561 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085564 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085566 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085569 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085571 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085573 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085576 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085580 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085582 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.085586 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086326 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 12:27:20.087409 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086333 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086336 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086341 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086345 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086348 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086351 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086354 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086358 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086361 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086363 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086366 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086369 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086372 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086375 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086377 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086380 2580 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086383 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086386 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086389 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086392 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 12:27:20.087891 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086395 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086398 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086400 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086403 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086406 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086408 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086411 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086414 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086417 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086420 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086422 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086441 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086445 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086448 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086450 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086453 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086456 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086459 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086462 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 12:27:20.088390 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086466 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086469 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086472 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086475 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086478 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086480 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086483 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086486 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086488 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086491 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086495 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086498 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086501 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086504 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086507 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086510 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086513 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086516 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086519 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 12:27:20.088876 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086521 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086528 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086531 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086534 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086537 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086539 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086542 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086545 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086548 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086550 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086553 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086556 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086559 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086561 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086564 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086566 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086569 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086572 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086574 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086577 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 12:27:20.089340 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086580 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086583 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086586 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086588 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086591 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086594 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086598 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086601 2580 feature_gate.go:328] unrecognized feature gate: Example Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086603 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086607 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086610 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086612 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086615 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086619 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086622 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086624 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086627 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086629 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086632 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.086635 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 12:27:20.089838 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087026 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087032 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087035 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087038 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087041 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087043 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087046 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087049 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087051 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087054 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087057 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087059 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087068 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087071 2580 feature_gate.go:328] unrecognized feature gate: Example Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087073 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087076 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087079 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087082 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087085 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087087 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 12:27:20.090331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087090 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087092 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087095 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087098 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087100 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087104 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087107 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087109 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087114 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087118 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087121 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087124 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087127 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087130 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087133 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087136 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087138 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087141 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087144 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 12:27:20.090820 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087146 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087149 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087151 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087154 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087156 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087159 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087162 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087165 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087168 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087171 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087173 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087176 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087178 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087181 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087184 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087186 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087189 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087192 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087195 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087198 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 12:27:20.091416 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087200 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087203 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087206 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087208 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087211 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087214 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087216 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087219 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087222 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087225 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087229 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087232 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087235 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087237 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087240 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087243 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087246 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087249 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087252 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 12:27:20.091966 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087256 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087259 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087261 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087264 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087267 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087270 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087272 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087275 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087279 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087282 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087285 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087289 2580 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087292 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087294 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087297 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087300 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087302 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087305 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087308 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087310 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 12:27:20.092449 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.087313 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087389 2580 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087397 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087406 2580 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087413 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087418 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087423 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087430 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087434 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087438 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087441 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087445 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087449 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087458 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087461 2580 flags.go:64] FLAG: --cgroup-root="" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087465 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087467 2580 flags.go:64] FLAG: --client-ca-file="" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087470 2580 flags.go:64] FLAG: --cloud-provider="external" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087473 2580 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087480 2580 flags.go:64] FLAG: --cluster-domain="" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087483 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087486 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087489 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087493 2580 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 12:27:20.092954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087497 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087501 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087504 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087507 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087510 2580 flags.go:64] FLAG: --contention-profiling="false" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087513 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087516 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087519 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087522 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087526 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087529 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087532 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087534 2580 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087538 2580 flags.go:64] FLAG: --enable-server="true" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087540 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087547 2580 flags.go:64] FLAG: --event-burst="100" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087550 2580 flags.go:64] FLAG: --event-qps="50" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087553 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087556 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087559 2580 flags.go:64] FLAG: --eviction-hard="" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087563 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087567 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087575 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087578 2580 flags.go:64] FLAG: --eviction-soft="" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087581 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 12:27:20.093556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087584 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087587 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087590 2580 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087593 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087596 2580 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087599 2580 flags.go:64] FLAG: --feature-gates="" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087603 2580 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087606 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087609 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087612 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087615 2580 flags.go:64] FLAG: --healthz-port="10248" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087618 2580 flags.go:64] FLAG: --help="false" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087621 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087624 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087627 2580 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087633 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087637 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087641 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087644 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087646 2580 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087649 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087652 2580 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087655 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087658 2580 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 12:27:20.094193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087661 2580 flags.go:64] FLAG: --kube-reserved="" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087664 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087667 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087670 2580 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087673 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087676 2580 flags.go:64] FLAG: --lock-file="" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087680 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087682 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087686 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087691 2580 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087694 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087696 2580 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087699 2580 flags.go:64] FLAG: --logging-format="text" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087702 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087705 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087708 2580 flags.go:64] FLAG: --manifest-url="" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087711 2580 flags.go:64] FLAG: --manifest-url-header="" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087715 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087718 2580 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087723 2580 flags.go:64] FLAG: --max-pods="110" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087726 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087729 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087732 2580 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087734 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087737 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 12:27:20.095217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087740 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087743 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087750 2580 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087753 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087756 2580 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087759 2580 flags.go:64] FLAG: --pod-cidr="" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087762 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087767 2580 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087770 2580 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087773 2580 flags.go:64] FLAG: --pods-per-core="0" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087776 2580 flags.go:64] FLAG: --port="10250" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087779 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087781 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04780821247b77759" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087785 2580 flags.go:64] FLAG: --qos-reserved="" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087788 2580 flags.go:64] FLAG: --read-only-port="10255" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087791 2580 flags.go:64] FLAG: --register-node="true" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087794 2580 flags.go:64] FLAG: --register-with-taints="" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087797 2580 flags.go:64] FLAG: --registry-burst="10" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087800 2580 flags.go:64] FLAG: --registry-qps="5" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087803 2580 flags.go:64] FLAG: --reserved-cpus="" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087806 2580 flags.go:64] FLAG: --reserved-memory="" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087809 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087812 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087815 2580 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087818 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 12:27:20.096296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087820 2580 flags.go:64] FLAG: --runonce="false" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087823 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087840 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087843 2580 flags.go:64] FLAG: --seccomp-default="false" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087846 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087849 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087852 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087855 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087858 2580 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087861 2580 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087864 2580 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087866 2580 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087869 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087872 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087875 2580 flags.go:64] FLAG: --system-cgroups="" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087878 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087887 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087890 2580 flags.go:64] FLAG: --tls-cert-file="" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087893 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087902 2580 flags.go:64] FLAG: --tls-min-version="" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087905 2580 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087908 2580 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087911 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087914 2580 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087917 2580 flags.go:64] FLAG: --v="2" Aug 04 12:27:20.097048 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087922 2580 flags.go:64] FLAG: --version="false" Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087926 2580 flags.go:64] FLAG: --vmodule="" Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087930 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.087934 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088064 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088068 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088071 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088074 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088077 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088080 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088082 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088085 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088087 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088090 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088093 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088095 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088098 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088101 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088104 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088106 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 12:27:20.097669 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088109 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088111 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088115 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088120 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088123 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088126 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088129 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088132 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088135 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088138 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088141 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088144 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088148 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088151 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088153 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088156 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088158 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088161 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088164 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 12:27:20.098180 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088166 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088169 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088171 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088174 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088177 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088179 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088182 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088184 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088189 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088191 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088194 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088197 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088199 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088202 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088204 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088207 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088213 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088216 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088219 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 12:27:20.098661 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088222 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088224 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088227 2580 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088230 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088234 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088237 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088241 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088244 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088246 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088249 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088251 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088254 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088257 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088259 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088262 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088265 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088267 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088270 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088272 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088275 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 12:27:20.099200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088278 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088280 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088283 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088285 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088288 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088291 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088293 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088296 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088298 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088302 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088305 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088307 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088310 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088312 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088315 2580 feature_gate.go:328] unrecognized feature gate: Example Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088318 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088320 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088323 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088327 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088329 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 12:27:20.099705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088332 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088335 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088337 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088340 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.088343 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.088351 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.097757 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.097775 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.098845 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.098854 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098926 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098932 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098935 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098939 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098942 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 12:27:20.100225 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098945 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098948 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098951 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098955 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098957 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098960 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098963 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098966 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098969 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098972 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098976 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098979 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098983 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098986 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098990 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098992 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098995 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.098999 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099002 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 12:27:20.100602 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099004 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099008 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099010 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099014 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099018 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099023 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099028 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099031 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099034 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099037 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099040 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099043 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099046 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099049 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099052 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099055 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099057 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099060 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099063 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 12:27:20.101081 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099065 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099068 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099070 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099073 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099076 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099079 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099082 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099085 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099087 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099090 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099093 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099095 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099098 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099101 2580 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099103 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099106 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099108 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099111 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099114 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099117 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 12:27:20.101565 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099120 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099122 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099125 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099127 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099130 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099132 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099135 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099138 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099140 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099143 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099145 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099148 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099150 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099153 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099156 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099158 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099167 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099170 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099173 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099176 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 12:27:20.102089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099178 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099181 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099183 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099186 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099188 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099191 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099193 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099196 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099199 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099203 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099206 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099208 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099212 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099214 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099217 2580 feature_gate.go:328] unrecognized feature gate: Example Aug 04 12:27:20.102574 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099219 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.099225 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099363 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099369 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099372 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099375 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099378 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099381 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099384 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099387 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099390 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099393 2580 feature_gate.go:328] unrecognized feature gate: Example Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099395 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099398 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099407 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 12:27:20.102962 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099410 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099413 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099416 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099418 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099421 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099424 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099427 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099430 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099432 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099435 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099437 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099440 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099443 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099446 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099448 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099451 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099454 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099457 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099459 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 12:27:20.103331 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099462 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099464 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099467 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099469 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099472 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099475 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099477 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099480 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099482 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099485 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099488 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099490 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099493 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099503 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099506 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099509 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099513 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099516 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099518 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 12:27:20.103801 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099521 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099523 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099526 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099529 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099531 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099534 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099537 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099540 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099543 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099545 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099548 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099550 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099553 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099555 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099558 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099561 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099563 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099566 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099568 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099571 2580 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 12:27:20.104291 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099573 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099576 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099579 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099581 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099583 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099588 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099591 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099600 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099603 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099606 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099608 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099611 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099613 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099616 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099618 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099620 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099623 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099626 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099628 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 12:27:20.104797 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099631 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099634 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099636 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099638 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099641 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099644 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099646 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099650 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:20.099653 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.099659 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.099787 2580 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.102222 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.104316 2580 server.go:1019] "Starting client certificate rotation" Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.104411 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 12:27:20.105265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.104444 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 12:27:20.107273 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.107258 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 12:27:20.108502 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.108481 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 12:27:20.121419 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.121401 2580 log.go:25] "Validated CRI v1 runtime API" Aug 04 12:27:20.124108 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.124095 2580 log.go:25] "Validated CRI v1 image API" Aug 04 12:27:20.124919 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.124906 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 12:27:20.127462 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.127438 2580 fs.go:135] Filesystem UUIDs: map[23872464-60ea-442b-a82e-8c0cf813d144:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7c0ce866-abcb-4f0b-a841-ca73011ced2c:/dev/nvme0n1p3] Aug 04 12:27:20.127531 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.127460 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 12:27:20.134288 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.134183 2580 manager.go:216] Machine: {Timestamp:2026-08-04 12:27:20.132712666 +0000 UTC m=+0.369144028 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098918 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29f41044be11575108be8c030ec7c8 SystemUUID:ec29f410-44be-1157-5108-be8c030ec7c8 BootID:47bf3bef-41ec-4f27-88c0-9703f7ead9e4 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1e:8e:58:0f:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1e:8e:58:0f:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:f1:c0:dd:f9:a3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 12:27:20.134288 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.134286 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 12:27:20.134400 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.134380 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 12:27:20.139037 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.139009 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 12:27:20.139486 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.139041 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-96.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 12:27:20.140484 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.140470 2580 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 12:27:20.140548 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.140490 2580 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 12:27:20.140548 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.140501 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 12:27:20.140548 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.140527 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 12:27:20.141762 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.141749 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 12:27:20.145780 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.145760 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 12:27:20.146042 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146028 2580 state_mem.go:36] "Initialized new in-memory state store" Aug 04 12:27:20.146188 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146177 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 12:27:20.146243 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146216 2580 kubelet.go:486] "Attempting to sync node with API server" Aug 04 12:27:20.146243 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146231 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 12:27:20.146334 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146246 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 12:27:20.146334 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146261 2580 kubelet.go:398] "Adding apiserver pod source" Aug 04 12:27:20.146334 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146273 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 12:27:20.146767 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146752 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 12:27:20.146858 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.146777 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 12:27:20.149105 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.149089 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 12:27:20.150959 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.150945 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 12:27:20.151005 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.150977 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 12:27:20.152567 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152556 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 12:27:20.152606 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152572 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 12:27:20.152606 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152579 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 12:27:20.152606 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152584 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 12:27:20.152606 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152593 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 12:27:20.152606 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152598 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 12:27:20.152606 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152607 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 12:27:20.152772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152612 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 12:27:20.152772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152619 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 12:27:20.152772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152625 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 12:27:20.152772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152633 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 12:27:20.152772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.152638 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 12:27:20.154566 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.154554 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 12:27:20.154616 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.154567 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 12:27:20.157591 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.157573 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-96.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 12:27:20.157863 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.157851 2580 server.go:1268] "Started kubelet" Aug 04 12:27:20.158002 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.157969 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 12:27:20.158080 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.158052 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 12:27:20.158136 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.158073 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 12:27:20.158210 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.158187 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 12:27:20.158343 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.158281 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 12:27:20.158723 ip-10-0-132-96 systemd[1]: Started Kubernetes Kubelet. Aug 04 12:27:20.159654 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.159639 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 12:27:20.160495 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.160480 2580 server.go:310] "Adding debug handlers to kubelet server" Aug 04 12:27:20.164293 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.164272 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 12:27:20.166377 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.166364 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 12:27:20.166448 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.166425 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 12:27:20.166707 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.166640 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.167417 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.167194 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 12:27:20.167417 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.167196 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 12:27:20.167417 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.167225 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 12:27:20.167417 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.167371 2580 reconstruct.go:97] "Volume reconstruction finished" Aug 04 12:27:20.167417 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.167381 2580 reconciler.go:29] "Reconciler: start to sync state" Aug 04 12:27:20.169635 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.169616 2580 factory.go:153] Registering CRI-O factory Aug 04 12:27:20.169723 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.169664 2580 factory.go:223] Registration of the crio container factory successfully Aug 04 12:27:20.169723 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.169716 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 12:27:20.169813 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.169727 2580 factory.go:55] Registering systemd factory Aug 04 12:27:20.169813 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.169735 2580 factory.go:223] Registration of the systemd container factory successfully Aug 04 12:27:20.169813 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.169755 2580 factory.go:103] Registering Raw factory Aug 04 12:27:20.169813 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.169768 2580 manager.go:1196] Started watching for new ooms in manager Aug 04 12:27:20.170536 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.170519 2580 manager.go:349] Starting recovery of all containers Aug 04 12:27:20.172377 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.172348 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 04 12:27:20.172530 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.172510 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 04 12:27:20.173562 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.172364 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-96.ec2.internal.18c899e021007eb9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-96.ec2.internal,UID:ip-10-0-132-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-96.ec2.internal,},FirstTimestamp:2026-08-04 12:27:20.157806265 +0000 UTC m=+0.394237627,LastTimestamp:2026-08-04 12:27:20.157806265 +0000 UTC m=+0.394237627,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-96.ec2.internal,}" Aug 04 12:27:20.183770 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.183640 2580 manager.go:354] Recovery completed Aug 04 12:27:20.188023 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.187982 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 12:27:20.191539 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.191516 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientMemory" Aug 04 12:27:20.191619 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.191557 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 12:27:20.191619 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.191569 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientPID" Aug 04 12:27:20.192077 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.192063 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rcrjn" Aug 04 12:27:20.192162 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.192147 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 12:27:20.192162 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.192159 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 12:27:20.192162 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.192162 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 12:27:20.192255 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.192178 2580 state_mem.go:36] "Initialized new in-memory state store" Aug 04 12:27:20.193272 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.193189 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-96.ec2.internal.18c899e0230339e8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-96.ec2.internal,UID:ip-10-0-132-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-96.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-96.ec2.internal,},FirstTimestamp:2026-08-04 12:27:20.191539688 +0000 UTC m=+0.427971052,LastTimestamp:2026-08-04 12:27:20.191539688 +0000 UTC m=+0.427971052,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-96.ec2.internal,}" Aug 04 12:27:20.193492 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.193476 2580 policy_none.go:49] "None policy: Start" Aug 04 12:27:20.193601 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.193498 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 12:27:20.193601 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.193513 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 12:27:20.193878 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.193860 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientMemory" Aug 04 12:27:20.193952 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.193894 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 12:27:20.193952 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.193909 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientPID" Aug 04 12:27:20.194584 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.194572 2580 policy_none.go:47] "Start" Aug 04 12:27:20.201959 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.201940 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rcrjn" Aug 04 12:27:20.203918 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.203819 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-96.ec2.internal.18c899e02303966f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-96.ec2.internal,UID:ip-10-0-132-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-96.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-96.ec2.internal,},FirstTimestamp:2026-08-04 12:27:20.191563375 +0000 UTC m=+0.427994735,LastTimestamp:2026-08-04 12:27:20.191563375 +0000 UTC m=+0.427994735,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-96.ec2.internal,}" Aug 04 12:27:20.227538 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.227504 2580 manager.go:337] "Starting Device Plugin manager" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.227544 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.227557 2580 server.go:87] "Starting device plugin registration server" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.227824 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.227858 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.227950 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.228068 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.228080 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.228669 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 12:27:20.236141 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.228712 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.269241 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.269203 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 12:27:20.270584 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.270567 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 12:27:20.270643 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.270602 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 12:27:20.270690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.270647 2580 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 12:27:20.270733 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.270691 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 12:27:20.276189 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.276169 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 12:27:20.328549 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.328469 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 12:27:20.329520 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.329495 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientMemory" Aug 04 12:27:20.329629 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.329530 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 12:27:20.329629 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.329553 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientPID" Aug 04 12:27:20.329629 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.329578 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.342358 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.342337 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.342441 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.342362 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-96.ec2.internal\": node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.371469 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.371445 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal"] Aug 04 12:27:20.371674 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.371505 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 12:27:20.372850 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.372815 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.372977 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.372965 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientMemory" Aug 04 12:27:20.373018 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.372992 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 12:27:20.373018 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.373002 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientPID" Aug 04 12:27:20.374106 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374095 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 12:27:20.374257 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374243 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.374301 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374271 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 12:27:20.374796 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374779 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientMemory" Aug 04 12:27:20.374796 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374795 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientMemory" Aug 04 12:27:20.374934 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374807 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 12:27:20.374934 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374816 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 12:27:20.374934 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374817 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientPID" Aug 04 12:27:20.374934 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.374845 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientPID" Aug 04 12:27:20.376060 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.376047 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.376108 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.376073 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 12:27:20.376737 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.376721 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientMemory" Aug 04 12:27:20.376816 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.376747 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 12:27:20.376816 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.376758 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientPID" Aug 04 12:27:20.390377 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.390357 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-96.ec2.internal\" not found" node="ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.394624 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.394610 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-96.ec2.internal\" not found" node="ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.469065 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.469039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/afdaddb70d01a23ead7adc4ab559b1de-config\") pod \"kube-apiserver-proxy-ip-10-0-132-96.ec2.internal\" (UID: \"afdaddb70d01a23ead7adc4ab559b1de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.469065 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.469068 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3de0da6bd019081fb7418e0b472bd7dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal\" (UID: \"3de0da6bd019081fb7418e0b472bd7dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.469274 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.469089 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de0da6bd019081fb7418e0b472bd7dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal\" (UID: \"3de0da6bd019081fb7418e0b472bd7dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.473132 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.473113 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.570049 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.570017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/afdaddb70d01a23ead7adc4ab559b1de-config\") pod \"kube-apiserver-proxy-ip-10-0-132-96.ec2.internal\" (UID: \"afdaddb70d01a23ead7adc4ab559b1de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.570143 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.570054 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3de0da6bd019081fb7418e0b472bd7dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal\" (UID: \"3de0da6bd019081fb7418e0b472bd7dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.570143 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.570076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de0da6bd019081fb7418e0b472bd7dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal\" (UID: \"3de0da6bd019081fb7418e0b472bd7dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.570143 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.570106 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de0da6bd019081fb7418e0b472bd7dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal\" (UID: \"3de0da6bd019081fb7418e0b472bd7dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.570143 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.570128 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3de0da6bd019081fb7418e0b472bd7dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal\" (UID: \"3de0da6bd019081fb7418e0b472bd7dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.570281 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.570134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/afdaddb70d01a23ead7adc4ab559b1de-config\") pod \"kube-apiserver-proxy-ip-10-0-132-96.ec2.internal\" (UID: \"afdaddb70d01a23ead7adc4ab559b1de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.574123 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.574106 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.674961 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.674885 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.693062 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.693039 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.697406 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:20.697193 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" Aug 04 12:27:20.775048 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.775011 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.875588 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.875553 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:20.976161 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:20.976073 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:21.031089 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.029903 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 12:27:21.076504 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:21.076476 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:21.102597 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.102557 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 12:27:21.102767 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.102744 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 12:27:21.102804 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.102746 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 12:27:21.177384 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:21.177357 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:21.208147 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.208113 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 12:22:20 +0000 UTC" deadline="2028-02-15 11:14:32.710283696 +0000 UTC" Aug 04 12:27:21.208147 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.208144 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13438h47m11.502143406s" Aug 04 12:27:21.252442 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.252357 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 12:27:21.253663 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.253644 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientMemory" Aug 04 12:27:21.253772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.253677 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 12:27:21.253772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.253688 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeHasSufficientPID" Aug 04 12:27:21.278444 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:21.278421 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:21.378919 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:21.378882 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:21.402985 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:21.402942 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3de0da6bd019081fb7418e0b472bd7dc.slice/crio-3754b1dc0b103cb60e5d9241fff0a418a6d561b35f24f23c09db990fef14c108 WatchSource:0}: Error finding container 3754b1dc0b103cb60e5d9241fff0a418a6d561b35f24f23c09db990fef14c108: Status 404 returned error can't find the container with id 3754b1dc0b103cb60e5d9241fff0a418a6d561b35f24f23c09db990fef14c108 Aug 04 12:27:21.403245 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:21.403226 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafdaddb70d01a23ead7adc4ab559b1de.slice/crio-609dc85068e90ee7feef866dfd7f2f4eac97241445b4102abd0a35ed2580cce6 WatchSource:0}: Error finding container 609dc85068e90ee7feef866dfd7f2f4eac97241445b4102abd0a35ed2580cce6: Status 404 returned error can't find the container with id 609dc85068e90ee7feef866dfd7f2f4eac97241445b4102abd0a35ed2580cce6 Aug 04 12:27:21.407306 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.407283 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 12:27:21.479034 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:21.478994 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:21.579595 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:21.579526 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:21.680376 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:21.680345 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-96.ec2.internal\" not found" Aug 04 12:27:21.702198 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.702173 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 12:27:21.767559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.767526 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" Aug 04 12:27:21.773987 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.773963 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 12:27:21.777874 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.777856 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 12:27:21.779519 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.779504 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" Aug 04 12:27:21.795207 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:21.795176 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 12:27:22.147603 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.147322 2580 apiserver.go:52] "Watching apiserver" Aug 04 12:27:22.155972 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.155944 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 12:27:22.156531 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.156500 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-rzfpc","openshift-multus/network-metrics-daemon-gbvjb","kube-system/global-pull-secret-syncer-tztk9","kube-system/konnectivity-agent-jj94z","openshift-image-registry/node-ca-wwggp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal","openshift-multus/multus-additional-cni-plugins-bv8vm","openshift-network-diagnostics/network-check-target-2hvfq","openshift-network-operator/iptables-alerter-srdgl","openshift-ovn-kubernetes/ovnkube-node-zp9xc","kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb","openshift-cluster-node-tuning-operator/tuned-skjkv","openshift-dns/node-resolver-vdjr9"] Aug 04 12:27:22.158324 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.158302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.159746 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.159721 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.160823 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.160800 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.160942 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.160906 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:22.161293 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.161269 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 12:27:22.161524 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.161452 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 12:27:22.161771 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.161731 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 12:27:22.162075 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.162058 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s5g82\"" Aug 04 12:27:22.162275 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.162260 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 12:27:22.162349 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.162277 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:22.162556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.162540 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bzl6k\"" Aug 04 12:27:22.162647 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.162631 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 12:27:22.162725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.162712 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 12:27:22.163292 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.163274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.164683 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.164659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.165066 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.165048 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n9t8g\"" Aug 04 12:27:22.165163 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.165073 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 12:27:22.165163 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.164658 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 12:27:22.165276 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.165202 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 12:27:22.165465 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.165445 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 12:27:22.165984 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.165775 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 12:27:22.166104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.166078 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6czjj\"" Aug 04 12:27:22.166206 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.166091 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 12:27:22.170151 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.166960 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 12:27:22.170773 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.167626 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 12:27:22.170918 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.167688 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 12:27:22.171034 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.167757 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q7ddl\"" Aug 04 12:27:22.171110 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.171065 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:22.171184 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.171160 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:22.171428 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.171408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.171958 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.171937 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.173696 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.173667 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.174079 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.174054 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 12:27:22.175211 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.175187 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 12:27:22.175310 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.175263 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 12:27:22.175379 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.175341 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-88sf5\"" Aug 04 12:27:22.175486 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.175467 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fvwcn\"" Aug 04 12:27:22.175553 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.175515 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 12:27:22.175938 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.175912 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:22.176105 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.175981 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:22.177115 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.177096 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 12:27:22.177228 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.177177 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 12:27:22.177630 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.177607 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.177729 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.177689 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 12:27:22.177729 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.177722 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 12:27:22.177841 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.177776 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bgvl4\"" Aug 04 12:27:22.178068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72828a2b-63bc-4568-8672-0b61671221d4-ovn-node-metrics-cert\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.178113 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-conf-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.178113 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178107 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 12:27:22.178180 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-daemon-config\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.178180 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178144 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddce6f29-037a-4430-a8d4-dda23d54f355-tmp\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.178180 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/77951ab4-2d3d-4f26-b3e7-04ad71384895-konnectivity-ca\") pod \"konnectivity-agent-jj94z\" (UID: \"77951ab4-2d3d-4f26-b3e7-04ad71384895\") " pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:22.178273 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.178273 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3967f0e1-aa24-4602-977c-c67b09b0974f-kubelet-config\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.178273 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-netns\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.178421 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178280 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-sys\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.178421 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-host\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.178421 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nbcx\" (UniqueName: \"kubernetes.io/projected/ddce6f29-037a-4430-a8d4-dda23d54f355-kube-api-access-6nbcx\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.178421 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-kubelet\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.178421 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178367 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-systemd\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.178421 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-cni-bin\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.178725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j77ph\" (UniqueName: \"kubernetes.io/projected/72828a2b-63bc-4568-8672-0b61671221d4-kube-api-access-j77ph\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.178725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cnibin\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.178725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178502 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-os-release\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.178725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43b4e662-0e13-433c-a7de-e198c6685657-serviceca\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.178725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178638 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlq7g\" (UniqueName: \"kubernetes.io/projected/84a09a0b-16f0-4b1d-86db-ce1f83984015-kube-api-access-wlq7g\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:22.178725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-cni-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.178725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-cni-multus\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178734 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b4e662-0e13-433c-a7de-e198c6685657-host\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178773 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgmdd\" (UniqueName: \"kubernetes.io/projected/4ae7801d-f289-447f-8932-37938fe6924c-kube-api-access-qgmdd\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178796 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-env-overrides\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178812 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-os-release\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178830 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-etc-kubernetes\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-var-lib-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178913 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-cni-netd\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-ovnkube-script-lib\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-cni-bin\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.178995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/77951ab4-2d3d-4f26-b3e7-04ad71384895-agent-certs\") pod \"konnectivity-agent-jj94z\" (UID: \"77951ab4-2d3d-4f26-b3e7-04ad71384895\") " pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-iptables-alerter-script\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txd6z\" (UniqueName: \"kubernetes.io/projected/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-kube-api-access-txd6z\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.179121 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179105 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-ovn\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-modprobe-d\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysconfig\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3967f0e1-aa24-4602-977c-c67b09b0974f-dbus\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ae7801d-f289-447f-8932-37938fe6924c-tmp-dir\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179285 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-run-netns\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179334 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-node-log\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179378 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-cnibin\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179470 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-kubernetes\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysctl-d\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-run\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179544 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crfzq\" (UniqueName: \"kubernetes.io/projected/43b4e662-0e13-433c-a7de-e198c6685657-kube-api-access-crfzq\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179570 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-multus-certs\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysctl-conf\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179665 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-lib-modules\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.179781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-var-lib-kubelet\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179733 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-slash\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179808 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvlxn\" (UniqueName: \"kubernetes.io/projected/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-kube-api-access-bvlxn\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-systemd-units\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-etc-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179913 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-ovnkube-config\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-system-cni-dir\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.179980 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-socket-dir-parent\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-k8s-cni-cncf-io\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-kubelet\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-systemd\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180109 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180133 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d5126a0-8ae6-40fe-8840-e034906ff40a-cni-binary-copy\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.180645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckxpv\" (UniqueName: \"kubernetes.io/projected/6d5126a0-8ae6-40fe-8840-e034906ff40a-kube-api-access-ckxpv\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.181467 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180232 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cni-binary-copy\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.181467 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ae7801d-f289-447f-8932-37938fe6924c-hosts-file\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.181467 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180296 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-host-slash\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.181467 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180321 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-log-socket\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.181467 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-system-cni-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.181467 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-hostroot\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.181467 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.180396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-tuned\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.181824 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.181549 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 12:27:22.182113 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.182093 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l6stk\"" Aug 04 12:27:22.182425 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.182389 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 12:27:22.182537 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.182521 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 12:27:22.182763 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.182743 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 12:27:22.187593 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.187570 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 12:27:22.209898 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.209863 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-htngb" Aug 04 12:27:22.220515 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.219218 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-htngb" Aug 04 12:27:22.268115 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.268085 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 12:27:22.275816 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.275762 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" event={"ID":"3de0da6bd019081fb7418e0b472bd7dc","Type":"ContainerStarted","Data":"3754b1dc0b103cb60e5d9241fff0a418a6d561b35f24f23c09db990fef14c108"} Aug 04 12:27:22.277245 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.277209 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" event={"ID":"afdaddb70d01a23ead7adc4ab559b1de","Type":"ContainerStarted","Data":"609dc85068e90ee7feef866dfd7f2f4eac97241445b4102abd0a35ed2580cce6"} Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-socket-dir-parent\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-k8s-cni-cncf-io\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-kubelet\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280656 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280683 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-socket-dir-parent\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280718 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-kubelet\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280692 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-k8s-cni-cncf-io\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280751 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-systemd\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280802 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280816 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-systemd\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d5126a0-8ae6-40fe-8840-e034906ff40a-cni-binary-copy\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckxpv\" (UniqueName: \"kubernetes.io/projected/6d5126a0-8ae6-40fe-8840-e034906ff40a-kube-api-access-ckxpv\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280875 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cni-binary-copy\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-device-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.280987 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:22.281438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.280992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ae7801d-f289-447f-8932-37938fe6924c-hosts-file\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.281052 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs podName:84a09a0b-16f0-4b1d-86db-ce1f83984015 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:22.781033426 +0000 UTC m=+3.017464777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs") pod "network-metrics-daemon-gbvjb" (UID: "84a09a0b-16f0-4b1d-86db-ce1f83984015") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281057 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-host-slash\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-log-socket\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281125 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-system-cni-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281154 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-hostroot\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281177 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ae7801d-f289-447f-8932-37938fe6924c-hosts-file\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281230 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-system-cni-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281227 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-host-slash\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-tuned\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281263 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-hostroot\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281259 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-log-socket\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281270 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72828a2b-63bc-4568-8672-0b61671221d4-ovn-node-metrics-cert\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-conf-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281370 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281462 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d5126a0-8ae6-40fe-8840-e034906ff40a-cni-binary-copy\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281556 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 12:27:22.282424 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-conf-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281762 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-daemon-config\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddce6f29-037a-4430-a8d4-dda23d54f355-tmp\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281823 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/77951ab4-2d3d-4f26-b3e7-04ad71384895-konnectivity-ca\") pod \"konnectivity-agent-jj94z\" (UID: \"77951ab4-2d3d-4f26-b3e7-04ad71384895\") " pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-socket-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3967f0e1-aa24-4602-977c-c67b09b0974f-kubelet-config\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-netns\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.281986 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-sys\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-host\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nbcx\" (UniqueName: \"kubernetes.io/projected/ddce6f29-037a-4430-a8d4-dda23d54f355-kube-api-access-6nbcx\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-kubelet\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282077 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-systemd\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-cni-bin\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j77ph\" (UniqueName: \"kubernetes.io/projected/72828a2b-63bc-4568-8672-0b61671221d4-kube-api-access-j77ph\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cnibin\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-os-release\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.283218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-registration-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-sys-fs\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43b4e662-0e13-433c-a7de-e198c6685657-serviceca\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282267 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-host\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlq7g\" (UniqueName: \"kubernetes.io/projected/84a09a0b-16f0-4b1d-86db-ce1f83984015-kube-api-access-wlq7g\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282306 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-daemon-config\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-cni-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-cni-multus\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282380 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b4e662-0e13-433c-a7de-e198c6685657-host\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgmdd\" (UniqueName: \"kubernetes.io/projected/4ae7801d-f289-447f-8932-37938fe6924c-kube-api-access-qgmdd\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-env-overrides\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-os-release\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-etc-kubernetes\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282556 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-var-lib-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-cni-netd\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-ovnkube-script-lib\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.284083 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-cni-bin\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282641 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43b4e662-0e13-433c-a7de-e198c6685657-serviceca\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282692 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-cni-bin\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/77951ab4-2d3d-4f26-b3e7-04ad71384895-agent-certs\") pod \"konnectivity-agent-jj94z\" (UID: \"77951ab4-2d3d-4f26-b3e7-04ad71384895\") " pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282747 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3967f0e1-aa24-4602-977c-c67b09b0974f-kubelet-config\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282768 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-iptables-alerter-script\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txd6z\" (UniqueName: \"kubernetes.io/projected/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-kube-api-access-txd6z\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282792 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-netns\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-ovn\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-modprobe-d\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282862 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-sys\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysconfig\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3967f0e1-aa24-4602-977c-c67b09b0974f-dbus\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.282963 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysconfig\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283006 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-cni-netd\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283022 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-multus-cni-dir\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283073 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-var-lib-cni-multus\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-kubelet\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283269 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-systemd\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283285 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/77951ab4-2d3d-4f26-b3e7-04ad71384895-konnectivity-ca\") pod \"konnectivity-agent-jj94z\" (UID: \"77951ab4-2d3d-4f26-b3e7-04ad71384895\") " pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283311 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-cni-bin\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283365 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b4e662-0e13-433c-a7de-e198c6685657-host\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283539 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-run-ovn\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283583 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-ovnkube-script-lib\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283655 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-modprobe-d\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-env-overrides\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283671 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-os-release\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283705 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ae7801d-f289-447f-8932-37938fe6924c-tmp-dir\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283729 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-os-release\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-run-netns\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283770 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cnibin\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283771 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3967f0e1-aa24-4602-977c-c67b09b0974f-dbus\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283822 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-etc-kubernetes\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.285946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-run-netns\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283873 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-node-log\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.283987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-cnibin\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-kubernetes\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysctl-d\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284084 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ae7801d-f289-447f-8932-37938fe6924c-tmp-dir\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-run\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284122 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-run\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crfzq\" (UniqueName: \"kubernetes.io/projected/43b4e662-0e13-433c-a7de-e198c6685657-kube-api-access-crfzq\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-multus-certs\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284229 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-node-log\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysctl-conf\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284203 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysctl-d\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284364 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-sysctl-conf\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284411 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-kubernetes\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.286893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284426 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-cnibin\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284448 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-iptables-alerter-script\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-lib-modules\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284495 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-var-lib-kubelet\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d5126a0-8ae6-40fe-8840-e034906ff40a-host-run-multus-certs\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284573 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-slash\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284636 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-var-lib-kubelet\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.284667 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvlxn\" (UniqueName: \"kubernetes.io/projected/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-kube-api-access-bvlxn\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284677 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-var-lib-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284713 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-slash\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.284737 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret podName:3967f0e1-aa24-4602-977c-c67b09b0974f nodeName:}" failed. No retries permitted until 2026-08-04 12:27:22.784718094 +0000 UTC m=+3.021149449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret") pod "global-pull-secret-syncer-tztk9" (UID: "3967f0e1-aa24-4602-977c-c67b09b0974f") : object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284756 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284840 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddce6f29-037a-4430-a8d4-dda23d54f355-lib-modules\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.287690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrx55\" (UniqueName: \"kubernetes.io/projected/a2d342d1-7203-49b8-a330-ca680aa105f4-kube-api-access-hrx55\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-systemd-units\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.284980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-etc-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.285006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-ovnkube-config\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.285031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-system-cni-dir\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.285115 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-systemd-units\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.285152 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72828a2b-63bc-4568-8672-0b61671221d4-etc-openvswitch\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.285185 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-system-cni-dir\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.285602 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72828a2b-63bc-4568-8672-0b61671221d4-ovnkube-config\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.285657 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddce6f29-037a-4430-a8d4-dda23d54f355-etc-tuned\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.286369 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72828a2b-63bc-4568-8672-0b61671221d4-ovn-node-metrics-cert\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.288014 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/77951ab4-2d3d-4f26-b3e7-04ad71384895-agent-certs\") pod \"konnectivity-agent-jj94z\" (UID: \"77951ab4-2d3d-4f26-b3e7-04ad71384895\") " pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:22.288559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.288234 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddce6f29-037a-4430-a8d4-dda23d54f355-tmp\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.289499 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.289474 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.289594 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.289474 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-cni-binary-copy\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.303439 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.303390 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txd6z\" (UniqueName: \"kubernetes.io/projected/bd30d00e-c7f7-4927-ac82-05deeeeaf37a-kube-api-access-txd6z\") pod \"iptables-alerter-srdgl\" (UID: \"bd30d00e-c7f7-4927-ac82-05deeeeaf37a\") " pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.303940 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.303878 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j77ph\" (UniqueName: \"kubernetes.io/projected/72828a2b-63bc-4568-8672-0b61671221d4-kube-api-access-j77ph\") pod \"ovnkube-node-zp9xc\" (UID: \"72828a2b-63bc-4568-8672-0b61671221d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.305404 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.305380 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckxpv\" (UniqueName: \"kubernetes.io/projected/6d5126a0-8ae6-40fe-8840-e034906ff40a-kube-api-access-ckxpv\") pod \"multus-rzfpc\" (UID: \"6d5126a0-8ae6-40fe-8840-e034906ff40a\") " pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.305701 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.305670 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvlxn\" (UniqueName: \"kubernetes.io/projected/960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5-kube-api-access-bvlxn\") pod \"multus-additional-cni-plugins-bv8vm\" (UID: \"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5\") " pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.305784 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.305674 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nbcx\" (UniqueName: \"kubernetes.io/projected/ddce6f29-037a-4430-a8d4-dda23d54f355-kube-api-access-6nbcx\") pod \"tuned-skjkv\" (UID: \"ddce6f29-037a-4430-a8d4-dda23d54f355\") " pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.306382 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.306356 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlq7g\" (UniqueName: \"kubernetes.io/projected/84a09a0b-16f0-4b1d-86db-ce1f83984015-kube-api-access-wlq7g\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:22.306883 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.306846 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crfzq\" (UniqueName: \"kubernetes.io/projected/43b4e662-0e13-433c-a7de-e198c6685657-kube-api-access-crfzq\") pod \"node-ca-wwggp\" (UID: \"43b4e662-0e13-433c-a7de-e198c6685657\") " pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.307331 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.307309 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgmdd\" (UniqueName: \"kubernetes.io/projected/4ae7801d-f289-447f-8932-37938fe6924c-kube-api-access-qgmdd\") pod \"node-resolver-vdjr9\" (UID: \"4ae7801d-f289-447f-8932-37938fe6924c\") " pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.386261 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-registration-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386261 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-sys-fs\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386509 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386290 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386509 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:22.386509 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386509 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386394 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-sys-fs\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386509 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386400 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386509 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386400 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-registration-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386409 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrx55\" (UniqueName: \"kubernetes.io/projected/a2d342d1-7203-49b8-a330-ca680aa105f4-kube-api-access-hrx55\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386546 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386562 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-device-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-socket-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386636 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-device-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.386725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.386725 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2d342d1-7203-49b8-a330-ca680aa105f4-socket-dir\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.398995 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.398890 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 12:27:22.398995 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.398946 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 12:27:22.398995 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.398962 2580 projected.go:196] Error preparing data for projected volume kube-api-access-ps5w5 for pod openshift-network-diagnostics/network-check-target-2hvfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:22.399266 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.399044 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5 podName:7d14c4a1-00fa-4508-8bd6-92fe8405a84e nodeName:}" failed. No retries permitted until 2026-08-04 12:27:22.899022618 +0000 UTC m=+3.135453986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ps5w5" (UniqueName: "kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5") pod "network-check-target-2hvfq" (UID: "7d14c4a1-00fa-4508-8bd6-92fe8405a84e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:22.403723 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.403680 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrx55\" (UniqueName: \"kubernetes.io/projected/a2d342d1-7203-49b8-a330-ca680aa105f4-kube-api-access-hrx55\") pod \"aws-ebs-csi-driver-node-m4nxb\" (UID: \"a2d342d1-7203-49b8-a330-ca680aa105f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.475174 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.475138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" Aug 04 12:27:22.487900 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.487865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-skjkv" Aug 04 12:27:22.499871 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.499850 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:22.505323 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.505299 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wwggp" Aug 04 12:27:22.515383 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.515357 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vdjr9" Aug 04 12:27:22.524309 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.524289 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rzfpc" Aug 04 12:27:22.535177 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.535138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-srdgl" Aug 04 12:27:22.543049 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.543032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:22.549850 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.549821 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" Aug 04 12:27:22.608054 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.608029 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 12:27:22.791146 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.791045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:22.791146 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.791103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:22.791381 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.791225 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:22.791381 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.791301 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret podName:3967f0e1-aa24-4602-977c-c67b09b0974f nodeName:}" failed. No retries permitted until 2026-08-04 12:27:23.791283265 +0000 UTC m=+4.027714619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret") pod "global-pull-secret-syncer-tztk9" (UID: "3967f0e1-aa24-4602-977c-c67b09b0974f") : object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:22.791381 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.791225 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:22.791502 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.791400 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs podName:84a09a0b-16f0-4b1d-86db-ce1f83984015 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:23.791382984 +0000 UTC m=+4.027814343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs") pod "network-metrics-daemon-gbvjb" (UID: "84a09a0b-16f0-4b1d-86db-ce1f83984015") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:22.992643 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:22.992597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:22.992831 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.992804 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 12:27:22.992831 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.992829 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 12:27:22.992945 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.992855 2580 projected.go:196] Error preparing data for projected volume kube-api-access-ps5w5 for pod openshift-network-diagnostics/network-check-target-2hvfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:22.992945 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:22.992919 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5 podName:7d14c4a1-00fa-4508-8bd6-92fe8405a84e nodeName:}" failed. No retries permitted until 2026-08-04 12:27:23.992904968 +0000 UTC m=+4.229336317 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ps5w5" (UniqueName: "kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5") pod "network-check-target-2hvfq" (UID: "7d14c4a1-00fa-4508-8bd6-92fe8405a84e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:23.132873 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:23.132777 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 12:27:23.221319 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:23.221276 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 12:22:22 +0000 UTC" deadline="2028-01-01 07:56:27.212359475 +0000 UTC" Aug 04 12:27:23.221319 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:23.221310 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12355h29m3.991054632s" Aug 04 12:27:23.271936 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:23.271897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:23.272098 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:23.272043 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:23.392933 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.392785 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b4e662_0e13_433c_a7de_e198c6685657.slice/crio-821892a2dd37b7332f793be36c446b56c40f72bd2fb0ac9c01735a848ac1af8a WatchSource:0}: Error finding container 821892a2dd37b7332f793be36c446b56c40f72bd2fb0ac9c01735a848ac1af8a: Status 404 returned error can't find the container with id 821892a2dd37b7332f793be36c446b56c40f72bd2fb0ac9c01735a848ac1af8a Aug 04 12:27:23.394538 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.394340 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77951ab4_2d3d_4f26_b3e7_04ad71384895.slice/crio-907764f3c394264b712005dadf98f37b5865a24137d8a8853b3ed945bd67171c WatchSource:0}: Error finding container 907764f3c394264b712005dadf98f37b5865a24137d8a8853b3ed945bd67171c: Status 404 returned error can't find the container with id 907764f3c394264b712005dadf98f37b5865a24137d8a8853b3ed945bd67171c Aug 04 12:27:23.397304 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.397280 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960ba42c_e184_4c44_b6c1_f5ec2ee9e3a5.slice/crio-e5f622896504ba895de364c523f42393392afbe41483fd0887173f64a2559f3f WatchSource:0}: Error finding container e5f622896504ba895de364c523f42393392afbe41483fd0887173f64a2559f3f: Status 404 returned error can't find the container with id e5f622896504ba895de364c523f42393392afbe41483fd0887173f64a2559f3f Aug 04 12:27:23.398158 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.398137 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d342d1_7203_49b8_a330_ca680aa105f4.slice/crio-efcedab41fa86ee429b1b3fd2e1f1cb5606e58d75fdc4d30d077357850083001 WatchSource:0}: Error finding container efcedab41fa86ee429b1b3fd2e1f1cb5606e58d75fdc4d30d077357850083001: Status 404 returned error can't find the container with id efcedab41fa86ee429b1b3fd2e1f1cb5606e58d75fdc4d30d077357850083001 Aug 04 12:27:23.400100 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.399801 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae7801d_f289_447f_8932_37938fe6924c.slice/crio-c65c1f75fb36156e10db4ee513e2c0d264b8e8d59d575cc71bb4af5a65a1886b WatchSource:0}: Error finding container c65c1f75fb36156e10db4ee513e2c0d264b8e8d59d575cc71bb4af5a65a1886b: Status 404 returned error can't find the container with id c65c1f75fb36156e10db4ee513e2c0d264b8e8d59d575cc71bb4af5a65a1886b Aug 04 12:27:23.401183 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.401131 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72828a2b_63bc_4568_8672_0b61671221d4.slice/crio-db0b396f3be85305b68d01d316dcfaf1287a54cf5e4cdc26ed535df8de010921 WatchSource:0}: Error finding container db0b396f3be85305b68d01d316dcfaf1287a54cf5e4cdc26ed535df8de010921: Status 404 returned error can't find the container with id db0b396f3be85305b68d01d316dcfaf1287a54cf5e4cdc26ed535df8de010921 Aug 04 12:27:23.402658 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.402557 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddce6f29_037a_4430_a8d4_dda23d54f355.slice/crio-1e2866e7a7dad056d01c5eedf6a093f20a57bc02871aa4dffa96326e33808d5d WatchSource:0}: Error finding container 1e2866e7a7dad056d01c5eedf6a093f20a57bc02871aa4dffa96326e33808d5d: Status 404 returned error can't find the container with id 1e2866e7a7dad056d01c5eedf6a093f20a57bc02871aa4dffa96326e33808d5d Aug 04 12:27:23.404832 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.403998 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd30d00e_c7f7_4927_ac82_05deeeeaf37a.slice/crio-b0dc33bf28d9ff66441da0b2830acae33554b6cdb4ff33b7df2dcd333d70fb6a WatchSource:0}: Error finding container b0dc33bf28d9ff66441da0b2830acae33554b6cdb4ff33b7df2dcd333d70fb6a: Status 404 returned error can't find the container with id b0dc33bf28d9ff66441da0b2830acae33554b6cdb4ff33b7df2dcd333d70fb6a Aug 04 12:27:23.405461 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:23.405438 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5126a0_8ae6_40fe_8840_e034906ff40a.slice/crio-4c046b3a39f9133bd8205bf0e010231edce97130e77ae3305a8e0a5cc845f28f WatchSource:0}: Error finding container 4c046b3a39f9133bd8205bf0e010231edce97130e77ae3305a8e0a5cc845f28f: Status 404 returned error can't find the container with id 4c046b3a39f9133bd8205bf0e010231edce97130e77ae3305a8e0a5cc845f28f Aug 04 12:27:23.801075 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:23.800866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:23.801243 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:23.801122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:23.801243 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:23.801006 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:23.801243 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:23.801206 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:23.801243 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:23.801231 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs podName:84a09a0b-16f0-4b1d-86db-ce1f83984015 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:25.801210634 +0000 UTC m=+6.037642279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs") pod "network-metrics-daemon-gbvjb" (UID: "84a09a0b-16f0-4b1d-86db-ce1f83984015") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:23.801385 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:23.801251 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret podName:3967f0e1-aa24-4602-977c-c67b09b0974f nodeName:}" failed. No retries permitted until 2026-08-04 12:27:25.801240246 +0000 UTC m=+6.037671597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret") pod "global-pull-secret-syncer-tztk9" (UID: "3967f0e1-aa24-4602-977c-c67b09b0974f") : object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:24.003515 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.003415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:24.003688 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:24.003630 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 12:27:24.003688 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:24.003651 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 12:27:24.003688 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:24.003664 2580 projected.go:196] Error preparing data for projected volume kube-api-access-ps5w5 for pod openshift-network-diagnostics/network-check-target-2hvfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:24.003888 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:24.003727 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5 podName:7d14c4a1-00fa-4508-8bd6-92fe8405a84e nodeName:}" failed. No retries permitted until 2026-08-04 12:27:26.003709535 +0000 UTC m=+6.240140888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ps5w5" (UniqueName: "kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5") pod "network-check-target-2hvfq" (UID: "7d14c4a1-00fa-4508-8bd6-92fe8405a84e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:24.221507 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.221437 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 12:22:22 +0000 UTC" deadline="2028-03-21 18:15:11.543330031 +0000 UTC" Aug 04 12:27:24.221507 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.221506 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14285h47m47.321828932s" Aug 04 12:27:24.271360 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.271273 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:24.271512 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:24.271424 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:24.271849 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.271819 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:24.271963 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:24.271932 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:24.306691 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.306641 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" event={"ID":"afdaddb70d01a23ead7adc4ab559b1de","Type":"ContainerStarted","Data":"6fedf928229600ebee24a83b8f28e78bd6348be41de37d983a7dfa77d021c458"} Aug 04 12:27:24.311152 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.311120 2580 generic.go:356] "Generic (PLEG): container finished" podID="3de0da6bd019081fb7418e0b472bd7dc" containerID="fbfcee7e40e0bbac6c13f8bc92a2608228c0ae083c8cf1ea6a426a06945002b4" exitCode=0 Aug 04 12:27:24.311284 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.311206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" event={"ID":"3de0da6bd019081fb7418e0b472bd7dc","Type":"ContainerDied","Data":"fbfcee7e40e0bbac6c13f8bc92a2608228c0ae083c8cf1ea6a426a06945002b4"} Aug 04 12:27:24.315294 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.315267 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rzfpc" event={"ID":"6d5126a0-8ae6-40fe-8840-e034906ff40a","Type":"ContainerStarted","Data":"4c046b3a39f9133bd8205bf0e010231edce97130e77ae3305a8e0a5cc845f28f"} Aug 04 12:27:24.318684 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.318662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-skjkv" event={"ID":"ddce6f29-037a-4430-a8d4-dda23d54f355","Type":"ContainerStarted","Data":"1e2866e7a7dad056d01c5eedf6a093f20a57bc02871aa4dffa96326e33808d5d"} Aug 04 12:27:24.324407 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.324376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vdjr9" event={"ID":"4ae7801d-f289-447f-8932-37938fe6924c","Type":"ContainerStarted","Data":"c65c1f75fb36156e10db4ee513e2c0d264b8e8d59d575cc71bb4af5a65a1886b"} Aug 04 12:27:24.333774 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.333735 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jj94z" event={"ID":"77951ab4-2d3d-4f26-b3e7-04ad71384895","Type":"ContainerStarted","Data":"907764f3c394264b712005dadf98f37b5865a24137d8a8853b3ed945bd67171c"} Aug 04 12:27:24.348994 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.347966 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-96.ec2.internal" podStartSLOduration=3.347947431 podStartE2EDuration="3.347947431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:24.327132356 +0000 UTC m=+4.563563738" watchObservedRunningTime="2026-08-04 12:27:24.347947431 +0000 UTC m=+4.584378805" Aug 04 12:27:24.354272 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.354198 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerStarted","Data":"e5f622896504ba895de364c523f42393392afbe41483fd0887173f64a2559f3f"} Aug 04 12:27:24.370690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.370615 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-srdgl" event={"ID":"bd30d00e-c7f7-4927-ac82-05deeeeaf37a","Type":"ContainerStarted","Data":"b0dc33bf28d9ff66441da0b2830acae33554b6cdb4ff33b7df2dcd333d70fb6a"} Aug 04 12:27:24.378533 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.378444 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"db0b396f3be85305b68d01d316dcfaf1287a54cf5e4cdc26ed535df8de010921"} Aug 04 12:27:24.384720 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.384588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" event={"ID":"a2d342d1-7203-49b8-a330-ca680aa105f4","Type":"ContainerStarted","Data":"efcedab41fa86ee429b1b3fd2e1f1cb5606e58d75fdc4d30d077357850083001"} Aug 04 12:27:24.394880 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:24.394851 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wwggp" event={"ID":"43b4e662-0e13-433c-a7de-e198c6685657","Type":"ContainerStarted","Data":"821892a2dd37b7332f793be36c446b56c40f72bd2fb0ac9c01735a848ac1af8a"} Aug 04 12:27:25.271298 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:25.271265 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:25.272059 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:25.271402 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:25.406183 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:25.406148 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" event={"ID":"3de0da6bd019081fb7418e0b472bd7dc","Type":"ContainerStarted","Data":"c9da4cefeeadbded560d8369e6dfdf423fae6a2071d30e66115f8aa8b6fe7424"} Aug 04 12:27:25.817843 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:25.817791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:25.818016 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:25.817900 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:25.818087 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:25.818024 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:25.818087 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:25.818078 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret podName:3967f0e1-aa24-4602-977c-c67b09b0974f nodeName:}" failed. No retries permitted until 2026-08-04 12:27:29.818060517 +0000 UTC m=+10.054491870 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret") pod "global-pull-secret-syncer-tztk9" (UID: "3967f0e1-aa24-4602-977c-c67b09b0974f") : object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:25.818466 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:25.818446 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:25.818537 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:25.818501 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs podName:84a09a0b-16f0-4b1d-86db-ce1f83984015 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:29.818485918 +0000 UTC m=+10.054917272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs") pod "network-metrics-daemon-gbvjb" (UID: "84a09a0b-16f0-4b1d-86db-ce1f83984015") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:26.020076 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:26.020036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:26.020258 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:26.020212 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 12:27:26.020258 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:26.020234 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 12:27:26.020258 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:26.020246 2580 projected.go:196] Error preparing data for projected volume kube-api-access-ps5w5 for pod openshift-network-diagnostics/network-check-target-2hvfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:26.020562 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:26.020307 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5 podName:7d14c4a1-00fa-4508-8bd6-92fe8405a84e nodeName:}" failed. No retries permitted until 2026-08-04 12:27:30.020288892 +0000 UTC m=+10.256720241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ps5w5" (UniqueName: "kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5") pod "network-check-target-2hvfq" (UID: "7d14c4a1-00fa-4508-8bd6-92fe8405a84e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:26.274081 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:26.273955 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:26.274572 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:26.274144 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:26.274572 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:26.274565 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:26.274699 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:26.274675 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:27.270877 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:27.270823 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:27.271076 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:27.270958 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:28.271247 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:28.271211 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:28.271247 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:28.271243 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:28.271768 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:28.271370 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:28.271768 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:28.271501 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:29.271259 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:29.271175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:29.271699 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:29.271289 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:29.851807 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:29.851767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:29.851977 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:29.851883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:29.851977 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:29.851911 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:29.851977 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:29.851976 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs podName:84a09a0b-16f0-4b1d-86db-ce1f83984015 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:37.851961729 +0000 UTC m=+18.088393078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs") pod "network-metrics-daemon-gbvjb" (UID: "84a09a0b-16f0-4b1d-86db-ce1f83984015") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:29.852098 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:29.851987 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:29.852098 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:29.852029 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret podName:3967f0e1-aa24-4602-977c-c67b09b0974f nodeName:}" failed. No retries permitted until 2026-08-04 12:27:37.852018155 +0000 UTC m=+18.088449507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret") pod "global-pull-secret-syncer-tztk9" (UID: "3967f0e1-aa24-4602-977c-c67b09b0974f") : object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:30.053177 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:30.053136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:30.053360 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:30.053338 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 12:27:30.053419 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:30.053372 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 12:27:30.053419 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:30.053388 2580 projected.go:196] Error preparing data for projected volume kube-api-access-ps5w5 for pod openshift-network-diagnostics/network-check-target-2hvfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:30.053537 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:30.053447 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5 podName:7d14c4a1-00fa-4508-8bd6-92fe8405a84e nodeName:}" failed. No retries permitted until 2026-08-04 12:27:38.053428691 +0000 UTC m=+18.289860039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ps5w5" (UniqueName: "kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5") pod "network-check-target-2hvfq" (UID: "7d14c4a1-00fa-4508-8bd6-92fe8405a84e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:30.272510 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:30.272431 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:30.272953 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:30.272541 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:30.272953 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:30.272603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:30.272953 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:30.272714 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:31.271211 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:31.271176 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:31.271407 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:31.271291 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:32.271641 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:32.271111 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:32.271641 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:32.271250 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:32.272218 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:32.271752 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:32.272218 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:32.271874 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:33.270947 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:33.270918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:33.271126 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:33.271037 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:34.273688 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:34.273647 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:34.274111 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:34.273789 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:34.274183 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:34.274157 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:34.274325 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:34.274240 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:35.272092 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:35.272057 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:35.272281 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:35.272194 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:36.271958 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:36.271929 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:36.272398 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:36.272080 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:36.272455 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:36.272415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:36.272536 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:36.272515 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:37.271308 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:37.271275 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:37.271527 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:37.271408 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:37.913045 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:37.913002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:37.913509 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:37.913057 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:37.913509 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:37.913155 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:37.913509 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:37.913163 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:37.913509 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:37.913207 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs podName:84a09a0b-16f0-4b1d-86db-ce1f83984015 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:53.913192596 +0000 UTC m=+34.149623943 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs") pod "network-metrics-daemon-gbvjb" (UID: "84a09a0b-16f0-4b1d-86db-ce1f83984015") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:37.913509 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:37.913232 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret podName:3967f0e1-aa24-4602-977c-c67b09b0974f nodeName:}" failed. No retries permitted until 2026-08-04 12:27:53.913213254 +0000 UTC m=+34.149644622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret") pod "global-pull-secret-syncer-tztk9" (UID: "3967f0e1-aa24-4602-977c-c67b09b0974f") : object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:38.114843 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:38.114791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:38.115009 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:38.114975 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 12:27:38.115009 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:38.115003 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 12:27:38.115124 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:38.115017 2580 projected.go:196] Error preparing data for projected volume kube-api-access-ps5w5 for pod openshift-network-diagnostics/network-check-target-2hvfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:38.115124 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:38.115082 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5 podName:7d14c4a1-00fa-4508-8bd6-92fe8405a84e nodeName:}" failed. No retries permitted until 2026-08-04 12:27:54.115066752 +0000 UTC m=+34.351498100 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ps5w5" (UniqueName: "kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5") pod "network-check-target-2hvfq" (UID: "7d14c4a1-00fa-4508-8bd6-92fe8405a84e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:38.271786 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:38.271697 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:38.271786 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:38.271748 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:38.272014 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:38.271862 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:38.272014 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:38.271966 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:39.270885 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:39.270851 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:39.271291 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:39.270970 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:40.272708 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:40.272675 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:40.273163 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:40.272762 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:40.273163 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:40.272784 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:40.273163 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:40.272907 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:41.271889 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:41.271849 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:41.272069 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:41.271991 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:42.272887 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.272383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:42.275719 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:42.274769 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:42.275719 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.275536 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:42.275719 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:42.275667 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:42.439944 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.439243 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rzfpc" event={"ID":"6d5126a0-8ae6-40fe-8840-e034906ff40a","Type":"ContainerStarted","Data":"a71ff68779077e002cf0573363804c81bd4521be3bbecbad9f476ab92f7acd68"} Aug 04 12:27:42.442166 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.442140 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-skjkv" event={"ID":"ddce6f29-037a-4430-a8d4-dda23d54f355","Type":"ContainerStarted","Data":"a51bed0e821898ee032129fb6ebdb168a396b39ddb339d0f6c632b61015efed9"} Aug 04 12:27:42.443723 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.443691 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vdjr9" event={"ID":"4ae7801d-f289-447f-8932-37938fe6924c","Type":"ContainerStarted","Data":"1279b92cdfd0407e2aead29a40f1e293098134e0ca03bb82cd3c5e5f513b74cd"} Aug 04 12:27:42.445404 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.445377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jj94z" event={"ID":"77951ab4-2d3d-4f26-b3e7-04ad71384895","Type":"ContainerStarted","Data":"8a099d6efe39bb6206d3321c1a35be30f3f0abde4a140bf70a859bbaac915fea"} Aug 04 12:27:42.447201 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.447086 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerStarted","Data":"6b98a1af37507424fb93f0bcfae53e90684c3241d7a2f8a18c38e32ec4590698"} Aug 04 12:27:42.449631 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.449560 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"1532cc990e30d09dec3530c762948d620737f99c44ce0fef3b1a52029528fbcd"} Aug 04 12:27:42.449631 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.449625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"f49b0aef71bea3c400eaa745b15266ac46a2e38c57e32f94a4170ecc68ac2a25"} Aug 04 12:27:42.451614 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.451557 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" event={"ID":"a2d342d1-7203-49b8-a330-ca680aa105f4","Type":"ContainerStarted","Data":"0bb4f4e13a24bbaaf2c4288ff003ca85e651007308feaf5809908aa489ba5c0e"} Aug 04 12:27:42.453009 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.452989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wwggp" event={"ID":"43b4e662-0e13-433c-a7de-e198c6685657","Type":"ContainerStarted","Data":"33d34c02c9a763c413b9b3c1d4c5af2b486c15778c99b51f9620b2ec18ead838"} Aug 04 12:27:42.462139 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.462088 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-96.ec2.internal" podStartSLOduration=20.462071582 podStartE2EDuration="20.462071582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:25.424846338 +0000 UTC m=+5.661277701" watchObservedRunningTime="2026-08-04 12:27:42.462071582 +0000 UTC m=+22.698502955" Aug 04 12:27:42.462866 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.462816 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rzfpc" podStartSLOduration=3.643367568 podStartE2EDuration="22.462803886s" totalImagesPullingTime="18.819436318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:42.461786879 +0000 UTC m=+22.698218250" watchObservedRunningTime="2026-08-04 12:27:42.462803886 +0000 UTC m=+22.699235247" Aug 04 12:27:42.510846 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.510787 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-skjkv" podStartSLOduration=3.781128097 podStartE2EDuration="22.510769694s" totalImagesPullingTime="18.729641597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:42.510467934 +0000 UTC m=+22.746899316" watchObservedRunningTime="2026-08-04 12:27:42.510769694 +0000 UTC m=+22.747201100" Aug 04 12:27:42.547032 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.546490 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jj94z" podStartSLOduration=12.430307708 podStartE2EDuration="22.546471534s" totalImagesPullingTime="10.116163826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:42.527967854 +0000 UTC m=+22.764399227" watchObservedRunningTime="2026-08-04 12:27:42.546471534 +0000 UTC m=+22.782902906" Aug 04 12:27:42.565501 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.565456 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wwggp" podStartSLOduration=3.8566917099999998 podStartE2EDuration="22.565437902s" totalImagesPullingTime="18.708746192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:42.54587501 +0000 UTC m=+22.782306380" watchObservedRunningTime="2026-08-04 12:27:42.565437902 +0000 UTC m=+22.801869272" Aug 04 12:27:42.565946 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:42.565923 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vdjr9" podStartSLOduration=3.833635716 podStartE2EDuration="22.565914715s" totalImagesPullingTime="18.732278999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:42.565513198 +0000 UTC m=+22.801944579" watchObservedRunningTime="2026-08-04 12:27:42.565914715 +0000 UTC m=+22.802346084" Aug 04 12:27:43.271539 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.271501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:43.271721 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:43.271642 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:43.394575 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.394555 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 12:27:43.455921 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.455858 2580 generic.go:356] "Generic (PLEG): container finished" podID="960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5" containerID="6b98a1af37507424fb93f0bcfae53e90684c3241d7a2f8a18c38e32ec4590698" exitCode=0 Aug 04 12:27:43.456046 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.455943 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerDied","Data":"6b98a1af37507424fb93f0bcfae53e90684c3241d7a2f8a18c38e32ec4590698"} Aug 04 12:27:43.459161 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.459131 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"8ad1b4ac4741dd5d19bd3a068fd933a61c6cdf682d5ac787dae25b30ccd821a1"} Aug 04 12:27:43.459272 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.459200 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"c4995f63838702a96821e7c8a8c3a9928fd032132b1701b69d4fe0747858c23e"} Aug 04 12:27:43.459272 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.459235 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"11c9b47a9390f83e2c0f112d2ccf0289c903f9deddd7e7dee4790981a5fcd7fb"} Aug 04 12:27:43.459366 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.459274 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"665e50831fac062056db079560feda926aa7a50dc49b4eb1aa50ef60780ed5af"} Aug 04 12:27:43.460795 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:43.460758 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" event={"ID":"a2d342d1-7203-49b8-a330-ca680aa105f4","Type":"ContainerStarted","Data":"d14311806da8dd6f433cf539dd10a6ce63919226cd3eafa1947d659343091e0f"} Aug 04 12:27:44.237811 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.237724 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T12:27:43.394572619Z","UUID":"988ec8e0-93f3-46c7-9ba0-4280c5f0ea69","Handler":null,"Name":"","Endpoint":""} Aug 04 12:27:44.239316 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.239294 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 12:27:44.239429 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.239338 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 12:27:44.271530 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.271503 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:44.271674 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:44.271637 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:44.272031 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.272012 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:44.272124 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:44.272103 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:44.464484 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.464449 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-srdgl" event={"ID":"bd30d00e-c7f7-4927-ac82-05deeeeaf37a","Type":"ContainerStarted","Data":"a8092b73404ce952d0cc0d81b2c6c10eba873900810c224c98e91d5312b21f5e"} Aug 04 12:27:44.466911 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.466879 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" event={"ID":"a2d342d1-7203-49b8-a330-ca680aa105f4","Type":"ContainerStarted","Data":"82a760f2322469f68a65641b3850eb273817d23b3facff4858bcc50041ec7d06"} Aug 04 12:27:44.504525 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.504441 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4nxb" podStartSLOduration=4.005644113 podStartE2EDuration="24.504426153s" totalImagesPullingTime="20.49878204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:44.504063482 +0000 UTC m=+24.740494851" watchObservedRunningTime="2026-08-04 12:27:44.504426153 +0000 UTC m=+24.740857522" Aug 04 12:27:44.504969 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:44.504937 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-srdgl" podStartSLOduration=5.775859594 podStartE2EDuration="24.504930019s" totalImagesPullingTime="18.729070425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:44.481862083 +0000 UTC m=+24.718293455" watchObservedRunningTime="2026-08-04 12:27:44.504930019 +0000 UTC m=+24.741361389" Aug 04 12:27:45.271120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:45.270935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:45.271292 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:45.271190 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:45.473280 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:45.473242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"8b64b6c30c5411c76e33ed898ce75c0a4ad5533b4568c4036b4c95674c85827f"} Aug 04 12:27:46.271933 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:46.271892 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:46.271933 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:46.271923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:46.272200 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:46.272041 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:46.272200 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:46.272174 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:46.364882 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:46.364850 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:46.365466 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:46.365440 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:47.271194 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:47.271119 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:47.271702 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:47.271259 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:48.149041 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.148855 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:48.149210 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.149112 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 04 12:27:48.149568 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.149547 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jj94z" Aug 04 12:27:48.271842 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.271799 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:48.272429 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:48.271961 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:48.272429 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.272013 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:48.272429 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:48.272096 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:48.481255 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.481162 2580 generic.go:356] "Generic (PLEG): container finished" podID="960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5" containerID="233b342916e61bc15253f311cba04993f138aa15c0beb8aa2bb0846ad1982186" exitCode=0 Aug 04 12:27:48.481404 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.481255 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerDied","Data":"233b342916e61bc15253f311cba04993f138aa15c0beb8aa2bb0846ad1982186"} Aug 04 12:27:48.485155 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.485127 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" event={"ID":"72828a2b-63bc-4568-8672-0b61671221d4","Type":"ContainerStarted","Data":"d0a7450410ea9cb60b0d77645edd8081d967e2dbca9ef1fdf52d580ccb01437a"} Aug 04 12:27:48.538849 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:48.538779 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" podStartSLOduration=9.764332032 podStartE2EDuration="28.538764203s" totalImagesPullingTime="18.774432171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:48.538256969 +0000 UTC m=+28.774688338" watchObservedRunningTime="2026-08-04 12:27:48.538764203 +0000 UTC m=+28.775195572" Aug 04 12:27:49.271359 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.271332 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:49.271732 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:49.271431 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:49.488506 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.488478 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:49.488966 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.488572 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:49.488966 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.488624 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:49.511065 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.511039 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:49.511170 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.511144 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:27:49.563805 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.563740 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tztk9"] Aug 04 12:27:49.563941 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.563903 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:49.564772 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:49.564037 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:49.564772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.564508 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2hvfq"] Aug 04 12:27:49.564772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.564609 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:49.564772 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:49.564735 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:49.565326 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.565303 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gbvjb"] Aug 04 12:27:49.565416 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:49.565402 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:49.565519 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:49.565497 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:50.492193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:50.492153 2580 generic.go:356] "Generic (PLEG): container finished" podID="960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5" containerID="4ba2b8c4925203d3dd90c4093c8ceb3b133eb2ad162fa05f3989956ea5b4809b" exitCode=0 Aug 04 12:27:50.492575 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:50.492238 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerDied","Data":"4ba2b8c4925203d3dd90c4093c8ceb3b133eb2ad162fa05f3989956ea5b4809b"} Aug 04 12:27:51.271114 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:51.271039 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:51.271114 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:51.271068 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:51.271288 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:51.271157 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:51.271288 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:51.271207 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:51.271351 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:51.271292 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:51.271392 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:51.271360 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:52.498231 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:52.498029 2580 generic.go:356] "Generic (PLEG): container finished" podID="960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5" containerID="afdb8d9aa0654a98ce441073f3b07ba596e8b27e6895b84ee0caa5f16ff17465" exitCode=0 Aug 04 12:27:52.498231 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:52.498111 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerDied","Data":"afdb8d9aa0654a98ce441073f3b07ba596e8b27e6895b84ee0caa5f16ff17465"} Aug 04 12:27:53.271226 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:53.271189 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:53.271359 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:53.271232 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:53.271359 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:53.271193 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:53.271359 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:53.271322 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tztk9" podUID="3967f0e1-aa24-4602-977c-c67b09b0974f" Aug 04 12:27:53.272356 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:53.271455 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvjb" podUID="84a09a0b-16f0-4b1d-86db-ce1f83984015" Aug 04 12:27:53.272356 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:53.271561 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2hvfq" podUID="7d14c4a1-00fa-4508-8bd6-92fe8405a84e" Aug 04 12:27:53.933151 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:53.933097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:53.933151 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:53.933153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:53.933925 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:53.933255 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:53.933925 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:53.933265 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:53.933925 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:53.933330 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret podName:3967f0e1-aa24-4602-977c-c67b09b0974f nodeName:}" failed. No retries permitted until 2026-08-04 12:28:25.933310753 +0000 UTC m=+66.169742105 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret") pod "global-pull-secret-syncer-tztk9" (UID: "3967f0e1-aa24-4602-977c-c67b09b0974f") : object "kube-system"/"original-pull-secret" not registered Aug 04 12:27:53.933925 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:53.933351 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs podName:84a09a0b-16f0-4b1d-86db-ce1f83984015 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:25.933341604 +0000 UTC m=+66.169772953 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs") pod "network-metrics-daemon-gbvjb" (UID: "84a09a0b-16f0-4b1d-86db-ce1f83984015") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 12:27:54.135332 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:54.135296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:54.135517 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:54.135476 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 12:27:54.135517 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:54.135497 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 12:27:54.135517 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:54.135507 2580 projected.go:196] Error preparing data for projected volume kube-api-access-ps5w5 for pod openshift-network-diagnostics/network-check-target-2hvfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:54.135644 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:54.135566 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5 podName:7d14c4a1-00fa-4508-8bd6-92fe8405a84e nodeName:}" failed. No retries permitted until 2026-08-04 12:28:26.13555339 +0000 UTC m=+66.371984738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ps5w5" (UniqueName: "kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5") pod "network-check-target-2hvfq" (UID: "7d14c4a1-00fa-4508-8bd6-92fe8405a84e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 12:27:55.014706 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.014637 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-96.ec2.internal" event="NodeReady" Aug 04 12:27:55.015068 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.014776 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 12:27:55.068712 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.068666 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-j56ss"] Aug 04 12:27:55.082506 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.082478 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx"] Aug 04 12:27:55.082694 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.082621 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.085485 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.085452 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.085485 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.085469 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Aug 04 12:27:55.085698 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.085462 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.085698 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.085651 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Aug 04 12:27:55.086036 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.086012 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6n66v\"" Aug 04 12:27:55.094760 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.094724 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b8f7cc49d-hpblp"] Aug 04 12:27:55.101304 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.101278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.105375 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.104267 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Aug 04 12:27:55.105375 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.105143 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.105375 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.105336 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.105595 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.105467 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-w5vzb\"" Aug 04 12:27:55.105595 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.105579 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Aug 04 12:27:55.106159 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.106079 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Aug 04 12:27:55.124666 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.124636 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4"] Aug 04 12:27:55.124824 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.124735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.129181 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.127704 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sgr6r\"" Aug 04 12:27:55.129181 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.127989 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 12:27:55.129181 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.128614 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 12:27:55.129181 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.128676 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 12:27:55.133482 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.133457 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 12:27:55.145002 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.144972 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-dtb6d"] Aug 04 12:27:55.145153 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.145102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4" Aug 04 12:27:55.147745 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.147719 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.147903 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.147863 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.148026 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.148004 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7fv44\"" Aug 04 12:27:55.163418 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.163373 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6"] Aug 04 12:27:55.163535 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.163475 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.165925 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.165904 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Aug 04 12:27:55.166045 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.165904 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.166045 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.165912 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Aug 04 12:27:55.166403 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.166371 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5pntd\"" Aug 04 12:27:55.166508 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.166496 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.173314 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.173257 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Aug 04 12:27:55.177457 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.177437 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g"] Aug 04 12:27:55.177563 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.177548 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.180168 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.180149 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-n4hv4\"" Aug 04 12:27:55.180319 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.180149 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.180467 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.180151 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Aug 04 12:27:55.180539 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.180166 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Aug 04 12:27:55.180598 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.180360 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.193739 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.193709 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp"] Aug 04 12:27:55.193901 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.193848 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g" Aug 04 12:27:55.196585 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.196564 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.196746 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.196567 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.196821 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.196806 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-w452h\"" Aug 04 12:27:55.205987 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.205967 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-j56ss"] Aug 04 12:27:55.206091 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.205999 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x"] Aug 04 12:27:55.206091 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.206078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:55.208774 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.208754 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.208916 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.208778 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.208916 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.208794 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Aug 04 12:27:55.208916 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.208813 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tmjdv\"" Aug 04 12:27:55.222258 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.222227 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-ccbd485dd-85zhv"] Aug 04 12:27:55.222394 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.222352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.225517 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.225493 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Aug 04 12:27:55.225646 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.225525 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.225646 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.225576 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.225876 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.225855 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-njtn2\"" Aug 04 12:27:55.226015 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.225861 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Aug 04 12:27:55.240545 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240517 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4"] Aug 04 12:27:55.240659 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240559 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx"] Aug 04 12:27:55.240659 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240581 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b8f7cc49d-hpblp"] Aug 04 12:27:55.240659 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240594 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.240779 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240601 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g"] Aug 04 12:27:55.240812 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240787 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6"] Aug 04 12:27:55.240875 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240818 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x"] Aug 04 12:27:55.240875 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240861 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-dtb6d"] Aug 04 12:27:55.241033 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.240996 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-ccbd485dd-85zhv"] Aug 04 12:27:55.241150 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.241041 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-69scq"] Aug 04 12:27:55.243152 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243128 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Aug 04 12:27:55.243152 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243135 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Aug 04 12:27:55.243314 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-bound-sa-token\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.243314 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6f75\" (UniqueName: \"kubernetes.io/projected/2ea55105-3c62-470a-b30d-667698ea32a2-kube-api-access-h6f75\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.243314 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhhvz\" (UniqueName: \"kubernetes.io/projected/5bf4ec3c-0d13-45e3-8cea-c2979f4fa689-kube-api-access-xhhvz\") pod \"volume-data-source-validator-5b8776669-bp4c4\" (UID: \"5bf4ec3c-0d13-45e3-8cea-c2979f4fa689\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4" Aug 04 12:27:55.243314 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b579baf-6a8c-441e-af03-1a66c1e1457c-ca-trust-extracted\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.243512 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63ed7430-8180-47e4-a254-6c34bcce3481-trusted-ca\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.243512 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6a350f36-d1a5-462b-8f92-eea0d0a47586-snapshots\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.243512 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-image-registry-private-configuration\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.243512 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243471 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.243512 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww5gf\" (UniqueName: \"kubernetes.io/projected/6a350f36-d1a5-462b-8f92-eea0d0a47586-kube-api-access-ww5gf\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.243752 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a350f36-d1a5-462b-8f92-eea0d0a47586-serving-cert\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.243752 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ea55105-3c62-470a-b30d-667698ea32a2-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.243752 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a350f36-d1a5-462b-8f92-eea0d0a47586-service-ca-bundle\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.243752 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.243752 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a350f36-d1a5-462b-8f92-eea0d0a47586-trusted-ca-bundle\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.243752 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243728 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.243752 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-certificates\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243731 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243784 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gwrvc\"" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/63ed7430-8180-47e4-a254-6c34bcce3481-config\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243904 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdlk9\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-kube-api-access-sdlk9\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.243972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-trusted-ca\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.244001 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-installation-pull-secrets\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.244027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63ed7430-8180-47e4-a254-6c34bcce3481-serving-cert\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.244056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.244104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.244084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt5xm\" (UniqueName: \"kubernetes.io/projected/63ed7430-8180-47e4-a254-6c34bcce3481-kube-api-access-tt5xm\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.244429 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.244139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a350f36-d1a5-462b-8f92-eea0d0a47586-tmp\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.261201 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.261169 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sj8xn"] Aug 04 12:27:55.261344 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.261298 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-69scq" Aug 04 12:27:55.264081 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.264042 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 12:27:55.264081 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.264058 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 12:27:55.264081 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.264044 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 12:27:55.264081 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.264072 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-25txl\"" Aug 04 12:27:55.277615 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.277585 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp"] Aug 04 12:27:55.277733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.277628 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sj8xn"] Aug 04 12:27:55.277733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.277651 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-69scq"] Aug 04 12:27:55.277733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.277660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:27:55.277733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.277666 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:27:55.277733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.277694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.278008 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.277882 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:27:55.280528 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.280507 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 12:27:55.280633 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.280575 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 12:27:55.280736 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.280721 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p4v7x\"" Aug 04 12:27:55.280781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.280756 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 12:27:55.280977 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.280961 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 12:27:55.281052 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.280968 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-st96r\"" Aug 04 12:27:55.281202 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.281190 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p2dbw\"" Aug 04 12:27:55.344516 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ea55105-3c62-470a-b30d-667698ea32a2-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.344687 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a350f36-d1a5-462b-8f92-eea0d0a47586-trusted-ca-bundle\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.344687 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd4x8\" (UniqueName: \"kubernetes.io/projected/5b4ada05-f9a1-4310-aee9-169f9584c2dc-kube-api-access-hd4x8\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.344687 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-trusted-ca\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.344687 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-installation-pull-secrets\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.344687 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344660 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f2035c4-94d3-4ae0-b90e-71e39bafac56-cert\") pod \"ingress-canary-69scq\" (UID: \"0f2035c4-94d3-4ae0-b90e-71e39bafac56\") " pod="openshift-ingress-canary/ingress-canary-69scq" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344691 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344715 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-bound-sa-token\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6f75\" (UniqueName: \"kubernetes.io/projected/2ea55105-3c62-470a-b30d-667698ea32a2-kube-api-access-h6f75\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344807 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpbkh\" (UniqueName: \"kubernetes.io/projected/773d5742-06cb-473c-a6d8-2cd2b6beef52-kube-api-access-wpbkh\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4qlk\" (UniqueName: \"kubernetes.io/projected/05382438-751b-4014-911d-cbd5b5db52cc-kube-api-access-n4qlk\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-default-certificate\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-image-registry-private-configuration\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww5gf\" (UniqueName: \"kubernetes.io/projected/6a350f36-d1a5-462b-8f92-eea0d0a47586-kube-api-access-ww5gf\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.344996 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.344970 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b4ada05-f9a1-4310-aee9-169f9584c2dc-config\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345018 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a350f36-d1a5-462b-8f92-eea0d0a47586-serving-cert\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npn99\" (UniqueName: \"kubernetes.io/projected/0f297761-617d-489b-b1f1-710bc3d8b11c-kube-api-access-npn99\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a350f36-d1a5-462b-8f92-eea0d0a47586-service-ca-bundle\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f297761-617d-489b-b1f1-710bc3d8b11c-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/63ed7430-8180-47e4-a254-6c34bcce3481-config\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f297761-617d-489b-b1f1-710bc3d8b11c-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345223 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-certificates\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b4ada05-f9a1-4310-aee9-169f9584c2dc-serving-cert\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdlk9\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-kube-api-access-sdlk9\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63ed7430-8180-47e4-a254-6c34bcce3481-serving-cert\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345317 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ea55105-3c62-470a-b30d-667698ea32a2-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt5xm\" (UniqueName: \"kubernetes.io/projected/63ed7430-8180-47e4-a254-6c34bcce3481-kube-api-access-tt5xm\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.345500 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a350f36-d1a5-462b-8f92-eea0d0a47586-tmp\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-stats-auth\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhvz\" (UniqueName: \"kubernetes.io/projected/5bf4ec3c-0d13-45e3-8cea-c2979f4fa689-kube-api-access-xhhvz\") pod \"volume-data-source-validator-5b8776669-bp4c4\" (UID: \"5bf4ec3c-0d13-45e3-8cea-c2979f4fa689\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63ed7430-8180-47e4-a254-6c34bcce3481-trusted-ca\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a350f36-d1a5-462b-8f92-eea0d0a47586-trusted-ca-bundle\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b579baf-6a8c-441e-af03-1a66c1e1457c-ca-trust-extracted\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345571 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6a350f36-d1a5-462b-8f92-eea0d0a47586-snapshots\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345595 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xr9t\" (UniqueName: \"kubernetes.io/projected/0f2035c4-94d3-4ae0-b90e-71e39bafac56-kube-api-access-8xr9t\") pod \"ingress-canary-69scq\" (UID: \"0f2035c4-94d3-4ae0-b90e-71e39bafac56\") " pod="openshift-ingress-canary/ingress-canary-69scq" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345617 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-trusted-ca\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.345632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqfps\" (UniqueName: \"kubernetes.io/projected/52a2532e-8336-469d-a31c-69c3abaf8784-kube-api-access-dqfps\") pod \"network-check-source-65bdbbff45-xzs9g\" (UID: \"52a2532e-8336-469d-a31c-69c3abaf8784\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.346019 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b579baf-6a8c-441e-af03-1a66c1e1457c-ca-trust-extracted\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.346075 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 12:27:55.346240 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.346155 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls podName:2ea55105-3c62-470a-b30d-667698ea32a2 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:55.846141526 +0000 UTC m=+36.082572874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-jchjx" (UID: "2ea55105-3c62-470a-b30d-667698ea32a2") : secret "cluster-monitoring-operator-tls" not found Aug 04 12:27:55.346768 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.346434 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6a350f36-d1a5-462b-8f92-eea0d0a47586-snapshots\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.346768 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.346713 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 12:27:55.346768 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.346729 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8f7cc49d-hpblp: secret "image-registry-tls" not found Aug 04 12:27:55.346926 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.346783 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls podName:1b579baf-6a8c-441e-af03-1a66c1e1457c nodeName:}" failed. No retries permitted until 2026-08-04 12:27:55.84676793 +0000 UTC m=+36.083199292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls") pod "image-registry-b8f7cc49d-hpblp" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c") : secret "image-registry-tls" not found Aug 04 12:27:55.347187 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.347159 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a350f36-d1a5-462b-8f92-eea0d0a47586-tmp\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.347316 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.347243 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63ed7430-8180-47e4-a254-6c34bcce3481-trusted-ca\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.347380 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.347318 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-certificates\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.347437 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.347410 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/63ed7430-8180-47e4-a254-6c34bcce3481-config\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.347720 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.347562 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a350f36-d1a5-462b-8f92-eea0d0a47586-service-ca-bundle\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.350186 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.350162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63ed7430-8180-47e4-a254-6c34bcce3481-serving-cert\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.350316 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.350296 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-installation-pull-secrets\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.350357 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.350296 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-image-registry-private-configuration\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.350357 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.350343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a350f36-d1a5-462b-8f92-eea0d0a47586-serving-cert\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.357283 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.357256 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww5gf\" (UniqueName: \"kubernetes.io/projected/6a350f36-d1a5-462b-8f92-eea0d0a47586-kube-api-access-ww5gf\") pod \"insights-operator-77696685b-j56ss\" (UID: \"6a350f36-d1a5-462b-8f92-eea0d0a47586\") " pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.357394 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.357263 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-bound-sa-token\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.357542 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.357524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6f75\" (UniqueName: \"kubernetes.io/projected/2ea55105-3c62-470a-b30d-667698ea32a2-kube-api-access-h6f75\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.357806 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.357782 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhhvz\" (UniqueName: \"kubernetes.io/projected/5bf4ec3c-0d13-45e3-8cea-c2979f4fa689-kube-api-access-xhhvz\") pod \"volume-data-source-validator-5b8776669-bp4c4\" (UID: \"5bf4ec3c-0d13-45e3-8cea-c2979f4fa689\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4" Aug 04 12:27:55.358171 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.358153 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt5xm\" (UniqueName: \"kubernetes.io/projected/63ed7430-8180-47e4-a254-6c34bcce3481-kube-api-access-tt5xm\") pod \"console-operator-5ccd6b445f-dtb6d\" (UID: \"63ed7430-8180-47e4-a254-6c34bcce3481\") " pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.358239 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.358222 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdlk9\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-kube-api-access-sdlk9\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.407314 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.407269 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-j56ss" Aug 04 12:27:55.446350 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446312 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd4x8\" (UniqueName: \"kubernetes.io/projected/5b4ada05-f9a1-4310-aee9-169f9584c2dc-kube-api-access-hd4x8\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.446497 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20f59554-f8ad-4737-9e31-21dc6849e377-config-volume\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.446589 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446560 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f2035c4-94d3-4ae0-b90e-71e39bafac56-cert\") pod \"ingress-canary-69scq\" (UID: \"0f2035c4-94d3-4ae0-b90e-71e39bafac56\") " pod="openshift-ingress-canary/ingress-canary-69scq" Aug 04 12:27:55.446742 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw72t\" (UniqueName: \"kubernetes.io/projected/20f59554-f8ad-4737-9e31-21dc6849e377-kube-api-access-zw72t\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.446742 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446622 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.446742 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446650 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:55.446742 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446680 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f59554-f8ad-4737-9e31-21dc6849e377-tmp-dir\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.446742 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpbkh\" (UniqueName: \"kubernetes.io/projected/773d5742-06cb-473c-a6d8-2cd2b6beef52-kube-api-access-wpbkh\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.446742 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.446734 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.446773 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.446807 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:55.946787217 +0000 UTC m=+36.183218567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : secret "router-metrics-certs-default" not found Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.446862 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls podName:05382438-751b-4014-911d-cbd5b5db52cc nodeName:}" failed. No retries permitted until 2026-08-04 12:27:55.946820964 +0000 UTC m=+36.183252313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jvvkp" (UID: "05382438-751b-4014-911d-cbd5b5db52cc") : secret "samples-operator-tls" not found Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4qlk\" (UniqueName: \"kubernetes.io/projected/05382438-751b-4014-911d-cbd5b5db52cc-kube-api-access-n4qlk\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-default-certificate\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b4ada05-f9a1-4310-aee9-169f9584c2dc-config\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npn99\" (UniqueName: \"kubernetes.io/projected/0f297761-617d-489b-b1f1-710bc3d8b11c-kube-api-access-npn99\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.446992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f297761-617d-489b-b1f1-710bc3d8b11c-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.447028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f297761-617d-489b-b1f1-710bc3d8b11c-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.447056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b4ada05-f9a1-4310-aee9-169f9584c2dc-serving-cert\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.447120 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.447118 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-stats-auth\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.447818 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.447476 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b4ada05-f9a1-4310-aee9-169f9584c2dc-config\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.447818 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.447489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.447818 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.447530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xr9t\" (UniqueName: \"kubernetes.io/projected/0f2035c4-94d3-4ae0-b90e-71e39bafac56-kube-api-access-8xr9t\") pod \"ingress-canary-69scq\" (UID: \"0f2035c4-94d3-4ae0-b90e-71e39bafac56\") " pod="openshift-ingress-canary/ingress-canary-69scq" Aug 04 12:27:55.447818 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.447567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqfps\" (UniqueName: \"kubernetes.io/projected/52a2532e-8336-469d-a31c-69c3abaf8784-kube-api-access-dqfps\") pod \"network-check-source-65bdbbff45-xzs9g\" (UID: \"52a2532e-8336-469d-a31c-69c3abaf8784\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g" Aug 04 12:27:55.447818 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.447594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.447818 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.447720 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:55.947640155 +0000 UTC m=+36.184071523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : configmap references non-existent config key: service-ca.crt Aug 04 12:27:55.448611 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.448588 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f297761-617d-489b-b1f1-710bc3d8b11c-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.449677 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.449654 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f297761-617d-489b-b1f1-710bc3d8b11c-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.449951 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.449936 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b4ada05-f9a1-4310-aee9-169f9584c2dc-serving-cert\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.450025 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.449940 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-stats-auth\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.450162 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.450140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f2035c4-94d3-4ae0-b90e-71e39bafac56-cert\") pod \"ingress-canary-69scq\" (UID: \"0f2035c4-94d3-4ae0-b90e-71e39bafac56\") " pod="openshift-ingress-canary/ingress-canary-69scq" Aug 04 12:27:55.451513 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.451495 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-default-certificate\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.455383 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.455211 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4" Aug 04 12:27:55.457028 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.456997 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd4x8\" (UniqueName: \"kubernetes.io/projected/5b4ada05-f9a1-4310-aee9-169f9584c2dc-kube-api-access-hd4x8\") pod \"service-ca-operator-784f97bcf6-xr9h6\" (UID: \"5b4ada05-f9a1-4310-aee9-169f9584c2dc\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.457391 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.457368 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpbkh\" (UniqueName: \"kubernetes.io/projected/773d5742-06cb-473c-a6d8-2cd2b6beef52-kube-api-access-wpbkh\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.457570 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.457548 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npn99\" (UniqueName: \"kubernetes.io/projected/0f297761-617d-489b-b1f1-710bc3d8b11c-kube-api-access-npn99\") pod \"kube-storage-version-migrator-operator-7878f4b88c-f2k9x\" (UID: \"0f297761-617d-489b-b1f1-710bc3d8b11c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.457798 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.457782 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4qlk\" (UniqueName: \"kubernetes.io/projected/05382438-751b-4014-911d-cbd5b5db52cc-kube-api-access-n4qlk\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:55.459546 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.459527 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqfps\" (UniqueName: \"kubernetes.io/projected/52a2532e-8336-469d-a31c-69c3abaf8784-kube-api-access-dqfps\") pod \"network-check-source-65bdbbff45-xzs9g\" (UID: \"52a2532e-8336-469d-a31c-69c3abaf8784\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g" Aug 04 12:27:55.459716 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.459699 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xr9t\" (UniqueName: \"kubernetes.io/projected/0f2035c4-94d3-4ae0-b90e-71e39bafac56-kube-api-access-8xr9t\") pod \"ingress-canary-69scq\" (UID: \"0f2035c4-94d3-4ae0-b90e-71e39bafac56\") " pod="openshift-ingress-canary/ingress-canary-69scq" Aug 04 12:27:55.476761 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.476742 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:27:55.487404 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.487383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" Aug 04 12:27:55.503582 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.503564 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g" Aug 04 12:27:55.532618 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.532588 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" Aug 04 12:27:55.548158 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.548130 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.548251 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.548188 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20f59554-f8ad-4737-9e31-21dc6849e377-config-volume\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.548251 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.548224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw72t\" (UniqueName: \"kubernetes.io/projected/20f59554-f8ad-4737-9e31-21dc6849e377-kube-api-access-zw72t\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.548348 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.548253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f59554-f8ad-4737-9e31-21dc6849e377-tmp-dir\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.548348 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.548286 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 12:27:55.548431 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.548352 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls podName:20f59554-f8ad-4737-9e31-21dc6849e377 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:56.048330531 +0000 UTC m=+36.284761881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls") pod "dns-default-sj8xn" (UID: "20f59554-f8ad-4737-9e31-21dc6849e377") : secret "dns-default-metrics-tls" not found Aug 04 12:27:55.548643 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.548624 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f59554-f8ad-4737-9e31-21dc6849e377-tmp-dir\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.548795 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.548780 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20f59554-f8ad-4737-9e31-21dc6849e377-config-volume\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.558584 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.558560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw72t\" (UniqueName: \"kubernetes.io/projected/20f59554-f8ad-4737-9e31-21dc6849e377-kube-api-access-zw72t\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:55.571464 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.571428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-69scq" Aug 04 12:27:55.850294 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.850258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:55.850525 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.850311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:55.850525 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.850433 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 12:27:55.850525 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.850452 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8f7cc49d-hpblp: secret "image-registry-tls" not found Aug 04 12:27:55.850525 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.850436 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 12:27:55.850525 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.850506 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls podName:1b579baf-6a8c-441e-af03-1a66c1e1457c nodeName:}" failed. No retries permitted until 2026-08-04 12:27:56.850491093 +0000 UTC m=+37.086922441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls") pod "image-registry-b8f7cc49d-hpblp" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c") : secret "image-registry-tls" not found Aug 04 12:27:55.850525 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.850522 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls podName:2ea55105-3c62-470a-b30d-667698ea32a2 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:56.850514004 +0000 UTC m=+37.086945353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-jchjx" (UID: "2ea55105-3c62-470a-b30d-667698ea32a2") : secret "cluster-monitoring-operator-tls" not found Aug 04 12:27:55.951459 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.951419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.951639 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.951490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:55.951639 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:55.951512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:55.951639 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.951613 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:56.951590251 +0000 UTC m=+37.188021599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : configmap references non-existent config key: service-ca.crt Aug 04 12:27:55.951639 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.951635 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 12:27:55.951878 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.951653 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 12:27:55.951878 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.951676 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls podName:05382438-751b-4014-911d-cbd5b5db52cc nodeName:}" failed. No retries permitted until 2026-08-04 12:27:56.951665168 +0000 UTC m=+37.188096516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jvvkp" (UID: "05382438-751b-4014-911d-cbd5b5db52cc") : secret "samples-operator-tls" not found Aug 04 12:27:55.951878 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:55.951696 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:56.95168232 +0000 UTC m=+37.188113668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : secret "router-metrics-certs-default" not found Aug 04 12:27:56.052195 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.052156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:56.052671 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.052326 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 12:27:56.052671 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.052390 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls podName:20f59554-f8ad-4737-9e31-21dc6849e377 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:57.052372709 +0000 UTC m=+37.288804056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls") pod "dns-default-sj8xn" (UID: "20f59554-f8ad-4737-9e31-21dc6849e377") : secret "dns-default-metrics-tls" not found Aug 04 12:27:56.871255 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.870292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:56.871255 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.870761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:56.871255 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.870935 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 12:27:56.871255 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.871012 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls podName:2ea55105-3c62-470a-b30d-667698ea32a2 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:58.870989019 +0000 UTC m=+39.107420370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-jchjx" (UID: "2ea55105-3c62-470a-b30d-667698ea32a2") : secret "cluster-monitoring-operator-tls" not found Aug 04 12:27:56.871648 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.870938 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 12:27:56.871648 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.871407 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8f7cc49d-hpblp: secret "image-registry-tls" not found Aug 04 12:27:56.871648 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.871461 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls podName:1b579baf-6a8c-441e-af03-1a66c1e1457c nodeName:}" failed. No retries permitted until 2026-08-04 12:27:58.871442958 +0000 UTC m=+39.107874329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls") pod "image-registry-b8f7cc49d-hpblp" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c") : secret "image-registry-tls" not found Aug 04 12:27:56.897929 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.897901 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6"] Aug 04 12:27:56.901473 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:56.901449 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4ada05_f9a1_4310_aee9_169f9584c2dc.slice/crio-07cd41da7fd30c8d967c668507d63201662467de01ddd88e081af34b24ef99a1 WatchSource:0}: Error finding container 07cd41da7fd30c8d967c668507d63201662467de01ddd88e081af34b24ef99a1: Status 404 returned error can't find the container with id 07cd41da7fd30c8d967c668507d63201662467de01ddd88e081af34b24ef99a1 Aug 04 12:27:56.904416 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.904379 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-dtb6d"] Aug 04 12:27:56.905941 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.905878 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g"] Aug 04 12:27:56.906352 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.906317 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4"] Aug 04 12:27:56.908427 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:56.908402 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a2532e_8336_469d_a31c_69c3abaf8784.slice/crio-8d924bf641cb870b840230aeb9f63cdb58432e268fe08b875eb2f45e4e15ec63 WatchSource:0}: Error finding container 8d924bf641cb870b840230aeb9f63cdb58432e268fe08b875eb2f45e4e15ec63: Status 404 returned error can't find the container with id 8d924bf641cb870b840230aeb9f63cdb58432e268fe08b875eb2f45e4e15ec63 Aug 04 12:27:56.909170 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:56.909149 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ed7430_8180_47e4_a254_6c34bcce3481.slice/crio-be11c7d88801dbf0978389061bad7cd1a805cbebb3dcb785ddbdabcf1d50b871 WatchSource:0}: Error finding container be11c7d88801dbf0978389061bad7cd1a805cbebb3dcb785ddbdabcf1d50b871: Status 404 returned error can't find the container with id be11c7d88801dbf0978389061bad7cd1a805cbebb3dcb785ddbdabcf1d50b871 Aug 04 12:27:56.912206 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:56.912192 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf4ec3c_0d13_45e3_8cea_c2979f4fa689.slice/crio-3940eeb3fa28f59ef98885db4becef25cf707bbcb8d6ede816a86930e0fdbbfd WatchSource:0}: Error finding container 3940eeb3fa28f59ef98885db4becef25cf707bbcb8d6ede816a86930e0fdbbfd: Status 404 returned error can't find the container with id 3940eeb3fa28f59ef98885db4becef25cf707bbcb8d6ede816a86930e0fdbbfd Aug 04 12:27:56.912914 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.912897 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7"] Aug 04 12:27:56.923875 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:27:56.923354 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a350f36_d1a5_462b_8f92_eea0d0a47586.slice/crio-ae1443ee3e34f6d27d83eca39f294fdbc0ce9e604167b7157058e9d75da87880 WatchSource:0}: Error finding container ae1443ee3e34f6d27d83eca39f294fdbc0ce9e604167b7157058e9d75da87880: Status 404 returned error can't find the container with id ae1443ee3e34f6d27d83eca39f294fdbc0ce9e604167b7157058e9d75da87880 Aug 04 12:27:56.926153 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.925393 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-j56ss"] Aug 04 12:27:56.926153 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.925429 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-69scq"] Aug 04 12:27:56.926153 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.925497 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x"] Aug 04 12:27:56.926153 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.925583 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:27:56.928627 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.927684 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7"] Aug 04 12:27:56.930614 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.929614 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hds44\"" Aug 04 12:27:56.930614 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.929878 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Aug 04 12:27:56.930614 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.930465 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Aug 04 12:27:56.971588 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.971207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:56.971588 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.971329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:56.971588 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:56.971362 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:56.971588 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.971541 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 12:27:56.971588 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.971554 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 12:27:56.971588 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.971577 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:58.9715575 +0000 UTC m=+39.207988864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : configmap references non-existent config key: service-ca.crt Aug 04 12:27:56.971991 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.971598 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls podName:05382438-751b-4014-911d-cbd5b5db52cc nodeName:}" failed. No retries permitted until 2026-08-04 12:27:58.971589078 +0000 UTC m=+39.208020427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jvvkp" (UID: "05382438-751b-4014-911d-cbd5b5db52cc") : secret "samples-operator-tls" not found Aug 04 12:27:56.971991 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:56.971613 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:58.971605619 +0000 UTC m=+39.208036989 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : secret "router-metrics-certs-default" not found Aug 04 12:27:57.072579 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.072546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:27:57.072945 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.072596 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c8ff1675-85c7-4e20-8404-3e4968add75c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:27:57.072945 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.072723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:57.072945 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:57.072869 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 12:27:57.072945 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:57.072921 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls podName:20f59554-f8ad-4737-9e31-21dc6849e377 nodeName:}" failed. No retries permitted until 2026-08-04 12:27:59.072907315 +0000 UTC m=+39.309338663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls") pod "dns-default-sj8xn" (UID: "20f59554-f8ad-4737-9e31-21dc6849e377") : secret "dns-default-metrics-tls" not found Aug 04 12:27:57.173808 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.173773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:27:57.173961 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.173821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c8ff1675-85c7-4e20-8404-3e4968add75c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:27:57.173961 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:57.173937 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 12:27:57.174075 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:57.174001 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert podName:c8ff1675-85c7-4e20-8404-3e4968add75c nodeName:}" failed. No retries permitted until 2026-08-04 12:27:57.673986507 +0000 UTC m=+37.910417855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-bfzn7" (UID: "c8ff1675-85c7-4e20-8404-3e4968add75c") : secret "networking-console-plugin-cert" not found Aug 04 12:27:57.174515 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.174493 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c8ff1675-85c7-4e20-8404-3e4968add75c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:27:57.515169 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.515025 2580 generic.go:356] "Generic (PLEG): container finished" podID="960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5" containerID="1df0076c740efabee5833ae2432b95dd9866bc66c50b55c8a3c7e874419377a1" exitCode=0 Aug 04 12:27:57.515169 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.515114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerDied","Data":"1df0076c740efabee5833ae2432b95dd9866bc66c50b55c8a3c7e874419377a1"} Aug 04 12:27:57.516607 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.516576 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" event={"ID":"0f297761-617d-489b-b1f1-710bc3d8b11c","Type":"ContainerStarted","Data":"3d08ba4cfe7acf5724dda24c7061526cc8e8005494b91bc80ea4511e4298a246"} Aug 04 12:27:57.518054 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.518026 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-j56ss" event={"ID":"6a350f36-d1a5-462b-8f92-eea0d0a47586","Type":"ContainerStarted","Data":"ae1443ee3e34f6d27d83eca39f294fdbc0ce9e604167b7157058e9d75da87880"} Aug 04 12:27:57.520170 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.520139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g" event={"ID":"52a2532e-8336-469d-a31c-69c3abaf8784","Type":"ContainerStarted","Data":"8d924bf641cb870b840230aeb9f63cdb58432e268fe08b875eb2f45e4e15ec63"} Aug 04 12:27:57.522195 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.522171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" event={"ID":"5b4ada05-f9a1-4310-aee9-169f9584c2dc","Type":"ContainerStarted","Data":"07cd41da7fd30c8d967c668507d63201662467de01ddd88e081af34b24ef99a1"} Aug 04 12:27:57.524435 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.524413 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4" event={"ID":"5bf4ec3c-0d13-45e3-8cea-c2979f4fa689","Type":"ContainerStarted","Data":"3940eeb3fa28f59ef98885db4becef25cf707bbcb8d6ede816a86930e0fdbbfd"} Aug 04 12:27:57.526017 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.525989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-69scq" event={"ID":"0f2035c4-94d3-4ae0-b90e-71e39bafac56","Type":"ContainerStarted","Data":"bf599c3e8f332e9e4ad8386f6736cc9f2616a9093cc05caf8919c9ac0609b4cd"} Aug 04 12:27:57.531669 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.531622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" event={"ID":"63ed7430-8180-47e4-a254-6c34bcce3481","Type":"ContainerStarted","Data":"be11c7d88801dbf0978389061bad7cd1a805cbebb3dcb785ddbdabcf1d50b871"} Aug 04 12:27:57.678991 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:57.678421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:27:57.678991 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:57.678545 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 12:27:57.678991 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:57.678602 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert podName:c8ff1675-85c7-4e20-8404-3e4968add75c nodeName:}" failed. No retries permitted until 2026-08-04 12:27:58.678583898 +0000 UTC m=+38.915015255 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-bfzn7" (UID: "c8ff1675-85c7-4e20-8404-3e4968add75c") : secret "networking-console-plugin-cert" not found Aug 04 12:27:58.548118 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:58.547138 2580 generic.go:356] "Generic (PLEG): container finished" podID="960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5" containerID="55e1d692dbd3b2fd7d54ff0138c20c8d37424eb2834671495056537fb9e1fa06" exitCode=0 Aug 04 12:27:58.548118 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:58.547219 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerDied","Data":"55e1d692dbd3b2fd7d54ff0138c20c8d37424eb2834671495056537fb9e1fa06"} Aug 04 12:27:58.693884 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:58.693610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:27:58.694094 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.694051 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 12:27:58.694164 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.694119 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert podName:c8ff1675-85c7-4e20-8404-3e4968add75c nodeName:}" failed. No retries permitted until 2026-08-04 12:28:00.694100416 +0000 UTC m=+40.930531776 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-bfzn7" (UID: "c8ff1675-85c7-4e20-8404-3e4968add75c") : secret "networking-console-plugin-cert" not found Aug 04 12:27:58.896082 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:58.896043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:27:58.896290 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:58.896115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:27:58.896346 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.896314 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 12:27:58.896390 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.896378 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls podName:2ea55105-3c62-470a-b30d-667698ea32a2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:02.896359674 +0000 UTC m=+43.132791025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-jchjx" (UID: "2ea55105-3c62-470a-b30d-667698ea32a2") : secret "cluster-monitoring-operator-tls" not found Aug 04 12:27:58.896468 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.896456 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 12:27:58.896524 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.896471 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8f7cc49d-hpblp: secret "image-registry-tls" not found Aug 04 12:27:58.896524 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.896504 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls podName:1b579baf-6a8c-441e-af03-1a66c1e1457c nodeName:}" failed. No retries permitted until 2026-08-04 12:28:02.89649288 +0000 UTC m=+43.132924230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls") pod "image-registry-b8f7cc49d-hpblp" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c") : secret "image-registry-tls" not found Aug 04 12:27:58.998168 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:58.997366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:58.998168 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:58.997419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:27:58.998168 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:58.997565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:27:58.998168 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.997747 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:02.997726743 +0000 UTC m=+43.234158096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : configmap references non-existent config key: service-ca.crt Aug 04 12:27:58.998168 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.997985 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 12:27:58.998168 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.998031 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls podName:05382438-751b-4014-911d-cbd5b5db52cc nodeName:}" failed. No retries permitted until 2026-08-04 12:28:02.998017357 +0000 UTC m=+43.234448708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jvvkp" (UID: "05382438-751b-4014-911d-cbd5b5db52cc") : secret "samples-operator-tls" not found Aug 04 12:27:58.998168 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.998083 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 12:27:58.998168 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:58.998137 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:02.998124981 +0000 UTC m=+43.234556332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : secret "router-metrics-certs-default" not found Aug 04 12:27:59.099401 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:59.098637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:27:59.099401 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:59.098972 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 12:27:59.099401 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:27:59.099039 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls podName:20f59554-f8ad-4737-9e31-21dc6849e377 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:03.099019816 +0000 UTC m=+43.335451169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls") pod "dns-default-sj8xn" (UID: "20f59554-f8ad-4737-9e31-21dc6849e377") : secret "dns-default-metrics-tls" not found Aug 04 12:27:59.434453 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:59.434339 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:27:59.527100 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:59.527011 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:27:59.556651 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:59.555910 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" event={"ID":"960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5","Type":"ContainerStarted","Data":"f5d785df9b3bd39788190c9011f7a6d3050cbd3ec4d14630644e81744530efe2"} Aug 04 12:27:59.827341 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:59.827293 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:27:59.887084 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:59.887008 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:27:59.950753 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:27:59.950720 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:00.343680 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:00.343650 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:00.716590 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:00.716510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:28:00.717000 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:00.716658 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 12:28:00.717000 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:00.716715 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert podName:c8ff1675-85c7-4e20-8404-3e4968add75c nodeName:}" failed. No retries permitted until 2026-08-04 12:28:04.716699642 +0000 UTC m=+44.953130992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-bfzn7" (UID: "c8ff1675-85c7-4e20-8404-3e4968add75c") : secret "networking-console-plugin-cert" not found Aug 04 12:28:02.933648 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:02.933610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:28:02.933648 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:02.933661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:28:02.934232 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:02.933769 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 12:28:02.934232 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:02.933775 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 12:28:02.934232 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:02.933794 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8f7cc49d-hpblp: secret "image-registry-tls" not found Aug 04 12:28:02.934232 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:02.933842 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls podName:2ea55105-3c62-470a-b30d-667698ea32a2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:10.933813003 +0000 UTC m=+51.170244351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-jchjx" (UID: "2ea55105-3c62-470a-b30d-667698ea32a2") : secret "cluster-monitoring-operator-tls" not found Aug 04 12:28:02.934232 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:02.933857 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls podName:1b579baf-6a8c-441e-af03-1a66c1e1457c nodeName:}" failed. No retries permitted until 2026-08-04 12:28:10.933850494 +0000 UTC m=+51.170281842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls") pod "image-registry-b8f7cc49d-hpblp" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c") : secret "image-registry-tls" not found Aug 04 12:28:02.992435 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:02.992379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:03.034925 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:03.034880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:03.035102 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:03.034992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:03.035102 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:03.035026 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:28:03.035102 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:03.035081 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:11.035055817 +0000 UTC m=+51.271487177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : configmap references non-existent config key: service-ca.crt Aug 04 12:28:03.035275 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:03.035149 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 12:28:03.035275 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:03.035165 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 12:28:03.035275 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:03.035203 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls podName:05382438-751b-4014-911d-cbd5b5db52cc nodeName:}" failed. No retries permitted until 2026-08-04 12:28:11.03518775 +0000 UTC m=+51.271619101 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jvvkp" (UID: "05382438-751b-4014-911d-cbd5b5db52cc") : secret "samples-operator-tls" not found Aug 04 12:28:03.035275 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:03.035226 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:11.035210121 +0000 UTC m=+51.271641472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : secret "router-metrics-certs-default" not found Aug 04 12:28:03.093240 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:03.093201 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:03.135878 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:03.135822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:28:03.136062 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:03.135989 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 12:28:03.136125 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:03.136064 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls podName:20f59554-f8ad-4737-9e31-21dc6849e377 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:11.136042892 +0000 UTC m=+51.372474256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls") pod "dns-default-sj8xn" (UID: "20f59554-f8ad-4737-9e31-21dc6849e377") : secret "dns-default-metrics-tls" not found Aug 04 12:28:04.704398 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:04.704366 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:04.750764 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:04.750742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:28:04.750916 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:04.750900 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 12:28:04.750972 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:04.750963 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert podName:c8ff1675-85c7-4e20-8404-3e4968add75c nodeName:}" failed. No retries permitted until 2026-08-04 12:28:12.750948067 +0000 UTC m=+52.987379415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-bfzn7" (UID: "c8ff1675-85c7-4e20-8404-3e4968add75c") : secret "networking-console-plugin-cert" not found Aug 04 12:28:04.765257 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:04.765231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:05.576024 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.575922 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" event={"ID":"0f297761-617d-489b-b1f1-710bc3d8b11c","Type":"ContainerStarted","Data":"066aac4155c398b4834644ec81976e12fb390111f93e67f578a97d6fe421e895"} Aug 04 12:28:05.577452 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.577421 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-j56ss" event={"ID":"6a350f36-d1a5-462b-8f92-eea0d0a47586","Type":"ContainerStarted","Data":"c3b6c3c6ec21aa2d15eb333a1eb6b0c7ff3b5b36933160d39bbb29ffd195d755"} Aug 04 12:28:05.579049 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.579019 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g" event={"ID":"52a2532e-8336-469d-a31c-69c3abaf8784","Type":"ContainerStarted","Data":"20d60186aa09f7a677e17da009748bea9d69324fcce8d3dd5820c7abb8592765"} Aug 04 12:28:05.580809 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.580649 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" event={"ID":"5b4ada05-f9a1-4310-aee9-169f9584c2dc","Type":"ContainerStarted","Data":"25ac65e80c836a3e727262ba0c4ef6c8f9c917c3917faed0d54e64a984b04387"} Aug 04 12:28:05.582151 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.582130 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4" event={"ID":"5bf4ec3c-0d13-45e3-8cea-c2979f4fa689","Type":"ContainerStarted","Data":"2f16a4b86ef70e8ab0e07a6d53d1faebfaa0baff8cf66d0c68e5ac16802ec8ce"} Aug 04 12:28:05.583998 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.583668 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-69scq" event={"ID":"0f2035c4-94d3-4ae0-b90e-71e39bafac56","Type":"ContainerStarted","Data":"52b0eb480b81dffff6f191478279f3e5f4dbd9dc1a0242b35b14fcc123978b21"} Aug 04 12:28:05.585406 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.585382 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/0.log" Aug 04 12:28:05.585505 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.585423 2580 generic.go:356] "Generic (PLEG): container finished" podID="63ed7430-8180-47e4-a254-6c34bcce3481" containerID="3dd5aa842dc5af3354c1bdee6530f8b4ceb7694b0dbc22fc4ec65ffe84431866" exitCode=255 Aug 04 12:28:05.585505 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.585463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" event={"ID":"63ed7430-8180-47e4-a254-6c34bcce3481","Type":"ContainerDied","Data":"3dd5aa842dc5af3354c1bdee6530f8b4ceb7694b0dbc22fc4ec65ffe84431866"} Aug 04 12:28:05.585679 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.585662 2580 scope.go:117] "RemoveContainer" containerID="3dd5aa842dc5af3354c1bdee6530f8b4ceb7694b0dbc22fc4ec65ffe84431866" Aug 04 12:28:05.596758 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.596672 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bv8vm" podStartSLOduration=16.419912269 podStartE2EDuration="45.596655732s" totalImagesPullingTime="29.176743463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:59.582716857 +0000 UTC m=+39.819148228" watchObservedRunningTime="2026-08-04 12:28:05.596655732 +0000 UTC m=+45.833087101" Aug 04 12:28:05.597249 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.597217 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" podStartSLOduration=30.885813581 podStartE2EDuration="38.597205152s" totalImagesPullingTime="7.711391571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:05.596307163 +0000 UTC m=+45.832738534" watchObservedRunningTime="2026-08-04 12:28:05.597205152 +0000 UTC m=+45.833636529" Aug 04 12:28:05.615164 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.615103 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-69scq" podStartSLOduration=2.904182674 podStartE2EDuration="10.615092024s" totalImagesPullingTime="7.71090935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:05.614893728 +0000 UTC m=+45.851325100" watchObservedRunningTime="2026-08-04 12:28:05.615092024 +0000 UTC m=+45.851523389" Aug 04 12:28:05.640680 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.640590 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-xzs9g" podStartSLOduration=30.653908684 podStartE2EDuration="38.640568154s" totalImagesPullingTime="7.98665947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:05.63979337 +0000 UTC m=+45.876224743" watchObservedRunningTime="2026-08-04 12:28:05.640568154 +0000 UTC m=+45.876999507" Aug 04 12:28:05.708054 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.707335 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-j56ss" podStartSLOduration=30.99454326 podStartE2EDuration="38.707313495s" totalImagesPullingTime="7.712770235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:05.687818716 +0000 UTC m=+45.924250086" watchObservedRunningTime="2026-08-04 12:28:05.707313495 +0000 UTC m=+45.943744845" Aug 04 12:28:05.708054 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.707501 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-bp4c4" podStartSLOduration=31.004682538 podStartE2EDuration="38.70749363s" totalImagesPullingTime="7.702811092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:05.705737313 +0000 UTC m=+45.942168686" watchObservedRunningTime="2026-08-04 12:28:05.70749363 +0000 UTC m=+45.943925001" Aug 04 12:28:05.725660 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.725185 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" podStartSLOduration=30.975459846 podStartE2EDuration="38.72516484s" totalImagesPullingTime="7.749704994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:05.724912724 +0000 UTC m=+45.961344094" watchObservedRunningTime="2026-08-04 12:28:05.72516484 +0000 UTC m=+45.961596210" Aug 04 12:28:05.808899 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.808861 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:05.875857 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:05.875757 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:06.018959 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.018921 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6mx47"] Aug 04 12:28:06.025551 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.025520 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:06.041242 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.041212 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6mx47"] Aug 04 12:28:06.041400 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.041353 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.045539 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.045132 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 12:28:06.045539 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.045230 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4bx8n\"" Aug 04 12:28:06.045539 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.045389 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 12:28:06.065297 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.065270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49832075-0827-4a88-97eb-c9653d2ba9b2-crio-socket\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.065429 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.065359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.065429 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.065403 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5f7t\" (UniqueName: \"kubernetes.io/projected/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-api-access-b5f7t\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.065533 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.065477 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.065533 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.065517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49832075-0827-4a88-97eb-c9653d2ba9b2-data-volume\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.133185 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.133075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:06.166390 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.166357 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49832075-0827-4a88-97eb-c9653d2ba9b2-crio-socket\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.166547 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.166442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.166547 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.166476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5f7t\" (UniqueName: \"kubernetes.io/projected/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-api-access-b5f7t\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.166547 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.166494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49832075-0827-4a88-97eb-c9653d2ba9b2-crio-socket\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.166547 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.166519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.166547 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.166546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49832075-0827-4a88-97eb-c9653d2ba9b2-data-volume\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.166797 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.166578 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 12:28:06.166797 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.166637 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm podName:49832075-0827-4a88-97eb-c9653d2ba9b2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:06.666616551 +0000 UTC m=+46.903047906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mx47" (UID: "49832075-0827-4a88-97eb-c9653d2ba9b2") : configmap "kube-rbac-proxy" not found Aug 04 12:28:06.166797 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.166654 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 12:28:06.166797 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.166706 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls podName:49832075-0827-4a88-97eb-c9653d2ba9b2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:06.666690805 +0000 UTC m=+46.903122156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mx47" (UID: "49832075-0827-4a88-97eb-c9653d2ba9b2") : secret "insights-runtime-extractor-tls" not found Aug 04 12:28:06.167028 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.166880 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49832075-0827-4a88-97eb-c9653d2ba9b2-data-volume\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.182488 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.182464 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5f7t\" (UniqueName: \"kubernetes.io/projected/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-api-access-b5f7t\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.594851 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.594807 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:28:06.595317 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.595297 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/0.log" Aug 04 12:28:06.595388 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.595355 2580 generic.go:356] "Generic (PLEG): container finished" podID="63ed7430-8180-47e4-a254-6c34bcce3481" containerID="3f03fff7cc9628aacc17f7e45c012568c32ae048fbaccc9b43d87a8e8accf836" exitCode=255 Aug 04 12:28:06.595533 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.595494 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" event={"ID":"63ed7430-8180-47e4-a254-6c34bcce3481","Type":"ContainerDied","Data":"3f03fff7cc9628aacc17f7e45c012568c32ae048fbaccc9b43d87a8e8accf836"} Aug 04 12:28:06.595611 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.595581 2580 scope.go:117] "RemoveContainer" containerID="3dd5aa842dc5af3354c1bdee6530f8b4ceb7694b0dbc22fc4ec65ffe84431866" Aug 04 12:28:06.596496 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.596477 2580 scope.go:117] "RemoveContainer" containerID="3f03fff7cc9628aacc17f7e45c012568c32ae048fbaccc9b43d87a8e8accf836" Aug 04 12:28:06.596901 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.596756 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-dtb6d_openshift-console-operator(63ed7430-8180-47e4-a254-6c34bcce3481)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" podUID="63ed7430-8180-47e4-a254-6c34bcce3481" Aug 04 12:28:06.671227 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.671197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.671385 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.671314 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 12:28:06.671449 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.671399 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls podName:49832075-0827-4a88-97eb-c9653d2ba9b2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:07.671377679 +0000 UTC m=+47.907809030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mx47" (UID: "49832075-0827-4a88-97eb-c9653d2ba9b2") : secret "insights-runtime-extractor-tls" not found Aug 04 12:28:06.671677 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.671653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:06.671748 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.671713 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 12:28:06.671821 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:06.671806 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm podName:49832075-0827-4a88-97eb-c9653d2ba9b2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:07.671790108 +0000 UTC m=+47.908221475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mx47" (UID: "49832075-0827-4a88-97eb-c9653d2ba9b2") : configmap "kube-rbac-proxy" not found Aug 04 12:28:06.811948 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.811912 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:06.917459 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:06.917347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:07.008468 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.008437 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:07.200674 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.200582 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:07.303071 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.303021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:07.579723 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.579695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:07.599437 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.599411 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:28:07.599777 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.599761 2580 scope.go:117] "RemoveContainer" containerID="3f03fff7cc9628aacc17f7e45c012568c32ae048fbaccc9b43d87a8e8accf836" Aug 04 12:28:07.599978 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:07.599956 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-dtb6d_openshift-console-operator(63ed7430-8180-47e4-a254-6c34bcce3481)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" podUID="63ed7430-8180-47e4-a254-6c34bcce3481" Aug 04 12:28:07.651155 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.651120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:07.681415 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.681385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:07.681602 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.681485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:07.681602 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:07.681516 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 12:28:07.681602 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:07.681598 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm podName:49832075-0827-4a88-97eb-c9653d2ba9b2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:09.681578773 +0000 UTC m=+49.918010135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mx47" (UID: "49832075-0827-4a88-97eb-c9653d2ba9b2") : configmap "kube-rbac-proxy" not found Aug 04 12:28:07.681731 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:07.681636 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 12:28:07.681731 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:07.681694 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls podName:49832075-0827-4a88-97eb-c9653d2ba9b2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:09.68167777 +0000 UTC m=+49.918109120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mx47" (UID: "49832075-0827-4a88-97eb-c9653d2ba9b2") : secret "insights-runtime-extractor-tls" not found Aug 04 12:28:07.788244 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:07.788222 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vdjr9_4ae7801d-f289-447f-8932-37938fe6924c/dns-node-resolver/0.log" Aug 04 12:28:08.786193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:08.786165 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wwggp_43b4e662-0e13-433c-a7de-e198c6685657/node-ca/0.log" Aug 04 12:28:08.989092 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:08.989054 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:09.184422 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:09.184392 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:09.290010 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:09.289979 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:09.390446 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:09.390413 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-69scq_0f2035c4-94d3-4ae0-b90e-71e39bafac56/serve-healthcheck-canary/0.log" Aug 04 12:28:09.699620 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:09.699577 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:09.699823 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:09.699651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:09.699823 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:09.699811 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 12:28:09.699952 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:09.699927 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls podName:49832075-0827-4a88-97eb-c9653d2ba9b2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:13.699903815 +0000 UTC m=+53.936335165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mx47" (UID: "49832075-0827-4a88-97eb-c9653d2ba9b2") : secret "insights-runtime-extractor-tls" not found Aug 04 12:28:09.700265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:09.700247 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49832075-0827-4a88-97eb-c9653d2ba9b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:10.060627 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:10.060597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" limit=1024 Aug 04 12:28:11.011605 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:11.011556 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:28:11.011787 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:11.011622 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:28:11.011787 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.011719 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 12:28:11.011787 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.011739 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8f7cc49d-hpblp: secret "image-registry-tls" not found Aug 04 12:28:11.011929 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.011792 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls podName:1b579baf-6a8c-441e-af03-1a66c1e1457c nodeName:}" failed. No retries permitted until 2026-08-04 12:28:27.01177666 +0000 UTC m=+67.248208008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls") pod "image-registry-b8f7cc49d-hpblp" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c") : secret "image-registry-tls" not found Aug 04 12:28:11.011929 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.011724 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 12:28:11.011929 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.011877 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls podName:2ea55105-3c62-470a-b30d-667698ea32a2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:27.011863486 +0000 UTC m=+67.248294833 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-jchjx" (UID: "2ea55105-3c62-470a-b30d-667698ea32a2") : secret "cluster-monitoring-operator-tls" not found Aug 04 12:28:11.113000 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:11.112965 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:11.113415 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:11.113069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:11.113415 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:11.113092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:28:11.113415 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.113129 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:27.113109965 +0000 UTC m=+67.349541313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : configmap references non-existent config key: service-ca.crt Aug 04 12:28:11.113415 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.113195 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 12:28:11.113415 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.113195 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 12:28:11.113415 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.113255 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls podName:05382438-751b-4014-911d-cbd5b5db52cc nodeName:}" failed. No retries permitted until 2026-08-04 12:28:27.113240279 +0000 UTC m=+67.349671630 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jvvkp" (UID: "05382438-751b-4014-911d-cbd5b5db52cc") : secret "samples-operator-tls" not found Aug 04 12:28:11.113415 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.113274 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs podName:773d5742-06cb-473c-a6d8-2cd2b6beef52 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:27.11326721 +0000 UTC m=+67.349698558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs") pod "router-default-ccbd485dd-85zhv" (UID: "773d5742-06cb-473c-a6d8-2cd2b6beef52") : secret "router-metrics-certs-default" not found Aug 04 12:28:11.214296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:11.214247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:28:11.214471 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.214406 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 12:28:11.214471 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:11.214468 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls podName:20f59554-f8ad-4737-9e31-21dc6849e377 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:27.214453183 +0000 UTC m=+67.450884531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls") pod "dns-default-sj8xn" (UID: "20f59554-f8ad-4737-9e31-21dc6849e377") : secret "dns-default-metrics-tls" not found Aug 04 12:28:12.829317 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:12.829274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:28:12.829700 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:12.829411 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 12:28:12.829700 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:12.829465 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert podName:c8ff1675-85c7-4e20-8404-3e4968add75c nodeName:}" failed. No retries permitted until 2026-08-04 12:28:28.829451033 +0000 UTC m=+69.065882380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-bfzn7" (UID: "c8ff1675-85c7-4e20-8404-3e4968add75c") : secret "networking-console-plugin-cert" not found Aug 04 12:28:13.736929 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:13.736891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:13.737222 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:13.737069 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 12:28:13.737222 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:13.737145 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls podName:49832075-0827-4a88-97eb-c9653d2ba9b2 nodeName:}" failed. No retries permitted until 2026-08-04 12:28:21.737125367 +0000 UTC m=+61.973556716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mx47" (UID: "49832075-0827-4a88-97eb-c9653d2ba9b2") : secret "insights-runtime-extractor-tls" not found Aug 04 12:28:15.476845 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:15.476807 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:28:15.477438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:15.476936 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:28:15.477438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:15.477307 2580 scope.go:117] "RemoveContainer" containerID="3f03fff7cc9628aacc17f7e45c012568c32ae048fbaccc9b43d87a8e8accf836" Aug 04 12:28:15.477630 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:15.477496 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-dtb6d_openshift-console-operator(63ed7430-8180-47e4-a254-6c34bcce3481)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" podUID="63ed7430-8180-47e4-a254-6c34bcce3481" Aug 04 12:28:15.622926 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:15.622891 2580 scope.go:117] "RemoveContainer" containerID="3f03fff7cc9628aacc17f7e45c012568c32ae048fbaccc9b43d87a8e8accf836" Aug 04 12:28:15.623121 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:15.623096 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-dtb6d_openshift-console-operator(63ed7430-8180-47e4-a254-6c34bcce3481)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" podUID="63ed7430-8180-47e4-a254-6c34bcce3481" Aug 04 12:28:21.505972 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:21.505944 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp9xc" Aug 04 12:28:21.810142 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:21.810113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:21.812591 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:21.812564 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49832075-0827-4a88-97eb-c9653d2ba9b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mx47\" (UID: \"49832075-0827-4a88-97eb-c9653d2ba9b2\") " pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:21.957243 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:21.957208 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4bx8n\"" Aug 04 12:28:21.964341 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:21.964315 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6mx47" Aug 04 12:28:22.099929 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:22.099822 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6mx47"] Aug 04 12:28:22.102705 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:22.102679 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49832075_0827_4a88_97eb_c9653d2ba9b2.slice/crio-bfd9220668818763a33732bf26815ea166474c526908c3a67e9ce05c886980ed WatchSource:0}: Error finding container bfd9220668818763a33732bf26815ea166474c526908c3a67e9ce05c886980ed: Status 404 returned error can't find the container with id bfd9220668818763a33732bf26815ea166474c526908c3a67e9ce05c886980ed Aug 04 12:28:22.644816 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:22.644783 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mx47" event={"ID":"49832075-0827-4a88-97eb-c9653d2ba9b2","Type":"ContainerStarted","Data":"e3a84caca769216f3930a0676a78a59b1326233a95c3816c8567a09dbc2e71f1"} Aug 04 12:28:22.645219 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:22.644852 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mx47" event={"ID":"49832075-0827-4a88-97eb-c9653d2ba9b2","Type":"ContainerStarted","Data":"bfd9220668818763a33732bf26815ea166474c526908c3a67e9ce05c886980ed"} Aug 04 12:28:23.650145 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:23.650100 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mx47" event={"ID":"49832075-0827-4a88-97eb-c9653d2ba9b2","Type":"ContainerStarted","Data":"ce50e11d58dd30de2eac3e3dd57a42f67dd5b3be5fae1281c336345788d31754"} Aug 04 12:28:24.656236 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:24.656204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mx47" event={"ID":"49832075-0827-4a88-97eb-c9653d2ba9b2","Type":"ContainerStarted","Data":"a1fbb6fb188b58b2357bd2db4b445fca4b511270dbbdc8f834bcdfc5745a40f7"} Aug 04 12:28:24.683789 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:24.683740 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6mx47" podStartSLOduration=17.65067343 podStartE2EDuration="19.683723936s" totalImagesPullingTime="2.033050506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:28:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:24.683553379 +0000 UTC m=+64.919984749" watchObservedRunningTime="2026-08-04 12:28:24.683723936 +0000 UTC m=+64.920155306" Aug 04 12:28:25.944690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:25.944655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:28:25.945098 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:25.944791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:28:25.948084 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:25.948068 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 12:28:25.948392 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:25.948377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 12:28:25.958114 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:25.958088 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a09a0b-16f0-4b1d-86db-ce1f83984015-metrics-certs\") pod \"network-metrics-daemon-gbvjb\" (UID: \"84a09a0b-16f0-4b1d-86db-ce1f83984015\") " pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:28:25.958219 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:25.958149 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3967f0e1-aa24-4602-977c-c67b09b0974f-original-pull-secret\") pod \"global-pull-secret-syncer-tztk9\" (UID: \"3967f0e1-aa24-4602-977c-c67b09b0974f\") " pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:28:26.147169 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.147134 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:28:26.149781 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.149755 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps5w5\" (UniqueName: \"kubernetes.io/projected/7d14c4a1-00fa-4508-8bd6-92fe8405a84e-kube-api-access-ps5w5\") pod \"network-check-target-2hvfq\" (UID: \"7d14c4a1-00fa-4508-8bd6-92fe8405a84e\") " pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:28:26.188260 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.188228 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tztk9" Aug 04 12:28:26.205970 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.205942 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-st96r\"" Aug 04 12:28:26.212784 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.212761 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p2dbw\"" Aug 04 12:28:26.213758 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.213741 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:28:26.220677 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.220656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvjb" Aug 04 12:28:26.340156 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.340113 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tztk9"] Aug 04 12:28:26.344061 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:26.344022 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3967f0e1_aa24_4602_977c_c67b09b0974f.slice/crio-e12a1572c09443529e23d1c150821b15768ce47a8451f422f4f0084fee0e6ef3 WatchSource:0}: Error finding container e12a1572c09443529e23d1c150821b15768ce47a8451f422f4f0084fee0e6ef3: Status 404 returned error can't find the container with id e12a1572c09443529e23d1c150821b15768ce47a8451f422f4f0084fee0e6ef3 Aug 04 12:28:26.372191 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.372163 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2hvfq"] Aug 04 12:28:26.375923 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:26.375887 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d14c4a1_00fa_4508_8bd6_92fe8405a84e.slice/crio-12aec729ea8e994d4468f81d639e0e473c5b6707382d63bf446e849512067597 WatchSource:0}: Error finding container 12aec729ea8e994d4468f81d639e0e473c5b6707382d63bf446e849512067597: Status 404 returned error can't find the container with id 12aec729ea8e994d4468f81d639e0e473c5b6707382d63bf446e849512067597 Aug 04 12:28:26.389454 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.389427 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gbvjb"] Aug 04 12:28:26.391988 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:26.391966 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a09a0b_16f0_4b1d_86db_ce1f83984015.slice/crio-0a7185089f4e85a54838543beb0820d1bbc7e1faf0b9226fd2e728a1b355e02a WatchSource:0}: Error finding container 0a7185089f4e85a54838543beb0820d1bbc7e1faf0b9226fd2e728a1b355e02a: Status 404 returned error can't find the container with id 0a7185089f4e85a54838543beb0820d1bbc7e1faf0b9226fd2e728a1b355e02a Aug 04 12:28:26.664134 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.664096 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tztk9" event={"ID":"3967f0e1-aa24-4602-977c-c67b09b0974f","Type":"ContainerStarted","Data":"e12a1572c09443529e23d1c150821b15768ce47a8451f422f4f0084fee0e6ef3"} Aug 04 12:28:26.665626 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.665590 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2hvfq" event={"ID":"7d14c4a1-00fa-4508-8bd6-92fe8405a84e","Type":"ContainerStarted","Data":"92dc799edb6cf1804b52f0f5c3e001f3ef94f01dd3c2eca40b2d7f144c9fd068"} Aug 04 12:28:26.665893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.665645 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:28:26.665893 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.665662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2hvfq" event={"ID":"7d14c4a1-00fa-4508-8bd6-92fe8405a84e","Type":"ContainerStarted","Data":"12aec729ea8e994d4468f81d639e0e473c5b6707382d63bf446e849512067597"} Aug 04 12:28:26.666799 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.666777 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gbvjb" event={"ID":"84a09a0b-16f0-4b1d-86db-ce1f83984015","Type":"ContainerStarted","Data":"0a7185089f4e85a54838543beb0820d1bbc7e1faf0b9226fd2e728a1b355e02a"} Aug 04 12:28:26.689448 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:26.689406 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2hvfq" podStartSLOduration=66.68939215 podStartE2EDuration="1m6.68939215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:26.688825668 +0000 UTC m=+66.925257038" watchObservedRunningTime="2026-08-04 12:28:26.68939215 +0000 UTC m=+66.925823519" Aug 04 12:28:27.056436 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.056393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:28:27.056947 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.056475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:28:27.059395 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.059361 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea55105-3c62-470a-b30d-667698ea32a2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-jchjx\" (UID: \"2ea55105-3c62-470a-b30d-667698ea32a2\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:28:27.059516 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.059425 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"image-registry-b8f7cc49d-hpblp\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:28:27.157155 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.157112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:27.157155 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.157163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:28:27.157411 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.157246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:27.157947 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.157906 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/773d5742-06cb-473c-a6d8-2cd2b6beef52-service-ca-bundle\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:27.160237 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.160212 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/773d5742-06cb-473c-a6d8-2cd2b6beef52-metrics-certs\") pod \"router-default-ccbd485dd-85zhv\" (UID: \"773d5742-06cb-473c-a6d8-2cd2b6beef52\") " pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:27.160345 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.160312 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/05382438-751b-4014-911d-cbd5b5db52cc-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jvvkp\" (UID: \"05382438-751b-4014-911d-cbd5b5db52cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:28:27.221412 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.221378 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-w5vzb\"" Aug 04 12:28:27.229234 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.229205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" Aug 04 12:28:27.240974 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.240938 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sgr6r\"" Aug 04 12:28:27.250763 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.247910 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:28:27.258982 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.258952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:28:27.261873 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.261816 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20f59554-f8ad-4737-9e31-21dc6849e377-metrics-tls\") pod \"dns-default-sj8xn\" (UID: \"20f59554-f8ad-4737-9e31-21dc6849e377\") " pod="openshift-dns/dns-default-sj8xn" Aug 04 12:28:27.272021 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.272000 2580 scope.go:117] "RemoveContainer" containerID="3f03fff7cc9628aacc17f7e45c012568c32ae048fbaccc9b43d87a8e8accf836" Aug 04 12:28:27.320307 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.320226 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tmjdv\"" Aug 04 12:28:27.327915 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.327887 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" Aug 04 12:28:27.354367 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.354338 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gwrvc\"" Aug 04 12:28:27.361033 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.361001 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:27.398975 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.398943 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p4v7x\"" Aug 04 12:28:27.406342 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.406130 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sj8xn" Aug 04 12:28:27.677906 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.677856 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:28:27.679527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.678873 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" event={"ID":"63ed7430-8180-47e4-a254-6c34bcce3481","Type":"ContainerStarted","Data":"fd28e0c9686aa6ad2594a03fcda5ee5e4ea0b9a5758424e966d53276fb3cc3be"} Aug 04 12:28:27.679985 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.679881 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:28:27.682709 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.681435 2580 patch_prober.go:28] interesting pod/console-operator-5ccd6b445f-dtb6d container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body= Aug 04 12:28:27.682709 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.681484 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" podUID="63ed7430-8180-47e4-a254-6c34bcce3481" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" Aug 04 12:28:27.701313 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.701258 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" podStartSLOduration=52.965167562 podStartE2EDuration="1m0.701240124s" totalImagesPullingTime="7.736072562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:27.700736816 +0000 UTC m=+67.937168190" watchObservedRunningTime="2026-08-04 12:28:27.701240124 +0000 UTC m=+67.937671495" Aug 04 12:28:27.713655 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.713488 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sj8xn"] Aug 04 12:28:27.729379 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.729351 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp"] Aug 04 12:28:27.754716 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.754684 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-ccbd485dd-85zhv"] Aug 04 12:28:27.774340 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.771543 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b8f7cc49d-hpblp"] Aug 04 12:28:27.873108 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:27.873030 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f59554_f8ad_4737_9e31_21dc6849e377.slice/crio-408e1e4a8e5f9e412db4c9b318ad5c7e5cceb7e8727d7f6a35da93789ad027fe WatchSource:0}: Error finding container 408e1e4a8e5f9e412db4c9b318ad5c7e5cceb7e8727d7f6a35da93789ad027fe: Status 404 returned error can't find the container with id 408e1e4a8e5f9e412db4c9b318ad5c7e5cceb7e8727d7f6a35da93789ad027fe Aug 04 12:28:27.874873 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:27.874614 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod773d5742_06cb_473c_a6d8_2cd2b6beef52.slice/crio-8e771485cebe2b7a89dc0fdb9c29068e0a8df44c3b5e8a3f02bffb377936258e WatchSource:0}: Error finding container 8e771485cebe2b7a89dc0fdb9c29068e0a8df44c3b5e8a3f02bffb377936258e: Status 404 returned error can't find the container with id 8e771485cebe2b7a89dc0fdb9c29068e0a8df44c3b5e8a3f02bffb377936258e Aug 04 12:28:27.875592 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:27.875544 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b579baf_6a8c_441e_af03_1a66c1e1457c.slice/crio-2e0bcc4a6480093744f1198612797c30be9c5c6d7afc93fc96ae8f1af2babb4b WatchSource:0}: Error finding container 2e0bcc4a6480093744f1198612797c30be9c5c6d7afc93fc96ae8f1af2babb4b: Status 404 returned error can't find the container with id 2e0bcc4a6480093744f1198612797c30be9c5c6d7afc93fc96ae8f1af2babb4b Aug 04 12:28:27.984517 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:27.984448 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx"] Aug 04 12:28:27.994200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:27.993752 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ea55105_3c62_470a_b30d_667698ea32a2.slice/crio-b8584f6670b5cb5ac05b11eb71e77e41bf1183eaa19c4d1c81c95f5a78007d8d WatchSource:0}: Error finding container b8584f6670b5cb5ac05b11eb71e77e41bf1183eaa19c4d1c81c95f5a78007d8d: Status 404 returned error can't find the container with id b8584f6670b5cb5ac05b11eb71e77e41bf1183eaa19c4d1c81c95f5a78007d8d Aug 04 12:28:28.686636 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.686588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" event={"ID":"05382438-751b-4014-911d-cbd5b5db52cc","Type":"ContainerStarted","Data":"77dd21ff156a15b0f50f82ba49a7264457aeba8d5e01b1cdfff35aa6f5e64952"} Aug 04 12:28:28.689342 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.689307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-ccbd485dd-85zhv" event={"ID":"773d5742-06cb-473c-a6d8-2cd2b6beef52","Type":"ContainerStarted","Data":"2b21e9c2745aeab958559d0f1c8b9611f187bc1f20032b01a205632783d96ad7"} Aug 04 12:28:28.689477 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.689366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-ccbd485dd-85zhv" event={"ID":"773d5742-06cb-473c-a6d8-2cd2b6beef52","Type":"ContainerStarted","Data":"8e771485cebe2b7a89dc0fdb9c29068e0a8df44c3b5e8a3f02bffb377936258e"} Aug 04 12:28:28.692681 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.692649 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gbvjb" event={"ID":"84a09a0b-16f0-4b1d-86db-ce1f83984015","Type":"ContainerStarted","Data":"9a7b35c24f75f6faebf29ae49d1cf2d75b3706b6f13206ccb7a70830fda313cc"} Aug 04 12:28:28.692786 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.692703 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gbvjb" event={"ID":"84a09a0b-16f0-4b1d-86db-ce1f83984015","Type":"ContainerStarted","Data":"1893ff652e2867a4853cd3afd0cda3d300a1520cc64151e71a81ce44e7b20707"} Aug 04 12:28:28.695293 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.694868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" event={"ID":"1b579baf-6a8c-441e-af03-1a66c1e1457c","Type":"ContainerStarted","Data":"8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623"} Aug 04 12:28:28.695293 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.694910 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" event={"ID":"1b579baf-6a8c-441e-af03-1a66c1e1457c","Type":"ContainerStarted","Data":"2e0bcc4a6480093744f1198612797c30be9c5c6d7afc93fc96ae8f1af2babb4b"} Aug 04 12:28:28.695293 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.695063 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:28:28.696672 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.696646 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" event={"ID":"2ea55105-3c62-470a-b30d-667698ea32a2","Type":"ContainerStarted","Data":"b8584f6670b5cb5ac05b11eb71e77e41bf1183eaa19c4d1c81c95f5a78007d8d"} Aug 04 12:28:28.699059 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.699032 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sj8xn" event={"ID":"20f59554-f8ad-4737-9e31-21dc6849e377","Type":"ContainerStarted","Data":"408e1e4a8e5f9e412db4c9b318ad5c7e5cceb7e8727d7f6a35da93789ad027fe"} Aug 04 12:28:28.705198 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.705169 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-dtb6d" Aug 04 12:28:28.714126 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.714079 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-ccbd485dd-85zhv" podStartSLOduration=61.714061499 podStartE2EDuration="1m1.714061499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:28.712731185 +0000 UTC m=+68.949162557" watchObservedRunningTime="2026-08-04 12:28:28.714061499 +0000 UTC m=+68.950492871" Aug 04 12:28:28.755301 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.754851 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gbvjb" podStartSLOduration=67.263969123 podStartE2EDuration="1m8.754813168s" totalImagesPullingTime="1.490844045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:28.734304227 +0000 UTC m=+68.970735597" watchObservedRunningTime="2026-08-04 12:28:28.754813168 +0000 UTC m=+68.991244540" Aug 04 12:28:28.780684 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.780631 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" podStartSLOduration=68.780611967 podStartE2EDuration="1m8.780611967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:28.780386575 +0000 UTC m=+69.016817954" watchObservedRunningTime="2026-08-04 12:28:28.780611967 +0000 UTC m=+69.017043338" Aug 04 12:28:28.877371 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.877331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:28:28.888516 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:28.881142 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c8ff1675-85c7-4e20-8404-3e4968add75c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-bfzn7\" (UID: \"c8ff1675-85c7-4e20-8404-3e4968add75c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:28:29.045587 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.045504 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hds44\"" Aug 04 12:28:29.052911 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.052884 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" Aug 04 12:28:29.245145 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.245085 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7"] Aug 04 12:28:29.249200 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:29.248971 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ff1675_85c7_4e20_8404_3e4968add75c.slice/crio-e7cb169f25a14d9a0c938bf10e4a29c36e0f04a37561b6d555a6cbd6a3880258 WatchSource:0}: Error finding container e7cb169f25a14d9a0c938bf10e4a29c36e0f04a37561b6d555a6cbd6a3880258: Status 404 returned error can't find the container with id e7cb169f25a14d9a0c938bf10e4a29c36e0f04a37561b6d555a6cbd6a3880258 Aug 04 12:28:29.364888 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.364797 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:29.367795 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.367771 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:29.496450 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.496419 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b"] Aug 04 12:28:29.501066 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.500997 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.524733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.522760 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 12:28:29.524733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.523815 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 12:28:29.524733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.524128 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Aug 04 12:28:29.524733 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.524231 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 12:28:29.530691 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.530658 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b"] Aug 04 12:28:29.584460 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.584415 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2qhh\" (UniqueName: \"kubernetes.io/projected/9146f37a-339d-4372-a1f0-4b32dcee4a10-kube-api-access-r2qhh\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.584635 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.584499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9146f37a-339d-4372-a1f0-4b32dcee4a10-tmp\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.584635 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.584592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9146f37a-339d-4372-a1f0-4b32dcee4a10-klusterlet-config\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.586245 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.586221 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-6vsh7"] Aug 04 12:28:29.588653 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.588630 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-6vsh7" Aug 04 12:28:29.593449 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.592477 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n7w27\"" Aug 04 12:28:29.593449 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.592721 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 04 12:28:29.593449 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.593291 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 04 12:28:29.627430 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.621931 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-6vsh7"] Aug 04 12:28:29.685341 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.685309 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9146f37a-339d-4372-a1f0-4b32dcee4a10-klusterlet-config\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.685517 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.685366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2qhh\" (UniqueName: \"kubernetes.io/projected/9146f37a-339d-4372-a1f0-4b32dcee4a10-kube-api-access-r2qhh\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.685517 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.685395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjb4z\" (UniqueName: \"kubernetes.io/projected/b6ddfc52-3897-42d6-87cd-f78774091ecb-kube-api-access-bjb4z\") pod \"downloads-77c974ffcf-6vsh7\" (UID: \"b6ddfc52-3897-42d6-87cd-f78774091ecb\") " pod="openshift-console/downloads-77c974ffcf-6vsh7" Aug 04 12:28:29.685517 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.685428 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9146f37a-339d-4372-a1f0-4b32dcee4a10-tmp\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.685907 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.685883 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9146f37a-339d-4372-a1f0-4b32dcee4a10-tmp\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.693056 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.691345 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9146f37a-339d-4372-a1f0-4b32dcee4a10-klusterlet-config\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.707403 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.707376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" event={"ID":"c8ff1675-85c7-4e20-8404-3e4968add75c","Type":"ContainerStarted","Data":"e7cb169f25a14d9a0c938bf10e4a29c36e0f04a37561b6d555a6cbd6a3880258"} Aug 04 12:28:29.707540 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.707517 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2qhh\" (UniqueName: \"kubernetes.io/projected/9146f37a-339d-4372-a1f0-4b32dcee4a10-kube-api-access-r2qhh\") pod \"klusterlet-addon-workmgr-7795b7d46-v868b\" (UID: \"9146f37a-339d-4372-a1f0-4b32dcee4a10\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.708212 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.708191 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:29.709569 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.709550 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-ccbd485dd-85zhv" Aug 04 12:28:29.791884 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.787069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjb4z\" (UniqueName: \"kubernetes.io/projected/b6ddfc52-3897-42d6-87cd-f78774091ecb-kube-api-access-bjb4z\") pod \"downloads-77c974ffcf-6vsh7\" (UID: \"b6ddfc52-3897-42d6-87cd-f78774091ecb\") " pod="openshift-console/downloads-77c974ffcf-6vsh7" Aug 04 12:28:29.807389 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.807330 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjb4z\" (UniqueName: \"kubernetes.io/projected/b6ddfc52-3897-42d6-87cd-f78774091ecb-kube-api-access-bjb4z\") pod \"downloads-77c974ffcf-6vsh7\" (UID: \"b6ddfc52-3897-42d6-87cd-f78774091ecb\") " pod="openshift-console/downloads-77c974ffcf-6vsh7" Aug 04 12:28:29.829147 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.829117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:29.903422 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:29.903332 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-6vsh7" Aug 04 12:28:32.556050 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:32.556018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b"] Aug 04 12:28:32.741089 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:32.740636 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9146f37a_339d_4372_a1f0_4b32dcee4a10.slice/crio-10d3b4498d4269c6c20589a622c6e2518a457a8656e492202f14f1a6006545a8 WatchSource:0}: Error finding container 10d3b4498d4269c6c20589a622c6e2518a457a8656e492202f14f1a6006545a8: Status 404 returned error can't find the container with id 10d3b4498d4269c6c20589a622c6e2518a457a8656e492202f14f1a6006545a8 Aug 04 12:28:32.902749 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:32.902721 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-6vsh7"] Aug 04 12:28:32.913401 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:32.913374 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ddfc52_3897_42d6_87cd_f78774091ecb.slice/crio-e243e8fa863aaa8b744df378514a3e80dc5fbb64983f07485a39483992a9c61d WatchSource:0}: Error finding container e243e8fa863aaa8b744df378514a3e80dc5fbb64983f07485a39483992a9c61d: Status 404 returned error can't find the container with id e243e8fa863aaa8b744df378514a3e80dc5fbb64983f07485a39483992a9c61d Aug 04 12:28:33.368163 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.368128 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr"] Aug 04 12:28:33.374397 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.374376 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" Aug 04 12:28:33.377062 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.377041 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Aug 04 12:28:33.377164 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.377145 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7kzwn\"" Aug 04 12:28:33.380279 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.380246 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr"] Aug 04 12:28:33.518447 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.518413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cb124e3c-963d-4201-be69-3ecb41e9a37e-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-w6dfr\" (UID: \"cb124e3c-963d-4201-be69-3ecb41e9a37e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" Aug 04 12:28:33.619251 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.619168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cb124e3c-963d-4201-be69-3ecb41e9a37e-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-w6dfr\" (UID: \"cb124e3c-963d-4201-be69-3ecb41e9a37e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" Aug 04 12:28:33.622268 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.622242 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cb124e3c-963d-4201-be69-3ecb41e9a37e-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-w6dfr\" (UID: \"cb124e3c-963d-4201-be69-3ecb41e9a37e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" Aug 04 12:28:33.684222 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.684184 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" Aug 04 12:28:33.727067 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.727031 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" event={"ID":"c8ff1675-85c7-4e20-8404-3e4968add75c","Type":"ContainerStarted","Data":"ad64cecb15c317b938b1453dbd5df914092ca43d65c4b17a55c71a43ec4b8582"} Aug 04 12:28:33.729484 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.729455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" event={"ID":"05382438-751b-4014-911d-cbd5b5db52cc","Type":"ContainerStarted","Data":"0e27c4e58ee390a90fecc2ed47ce39a6655f5193d2a7856f61d7f2053da864be"} Aug 04 12:28:33.729615 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.729511 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" event={"ID":"05382438-751b-4014-911d-cbd5b5db52cc","Type":"ContainerStarted","Data":"fbe7c64e80db9a605b078eebce21034e2cf516c351a57bf86f4b7a28aace217f"} Aug 04 12:28:33.731952 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.731887 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tztk9" event={"ID":"3967f0e1-aa24-4602-977c-c67b09b0974f","Type":"ContainerStarted","Data":"384c8327f1845d622c180ae076e0933b3dde0c41557762ff401b98c1757b59f7"} Aug 04 12:28:33.734074 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.734053 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" event={"ID":"2ea55105-3c62-470a-b30d-667698ea32a2","Type":"ContainerStarted","Data":"3b83d69b89aa3ff1836c384bad57b39f74275a5112dac9bdf6af377474984cf0"} Aug 04 12:28:33.736296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.736193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sj8xn" event={"ID":"20f59554-f8ad-4737-9e31-21dc6849e377","Type":"ContainerStarted","Data":"40c8f2a5ece9af2942c9c699af3f534c7453dbcb0b272bc99852d89e9b224722"} Aug 04 12:28:33.736296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.736241 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sj8xn" event={"ID":"20f59554-f8ad-4737-9e31-21dc6849e377","Type":"ContainerStarted","Data":"2391710131fbcf468a87a21529299cc61762b270e4068412d183729f7c963e2c"} Aug 04 12:28:33.736493 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.736474 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sj8xn" Aug 04 12:28:33.737629 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.737604 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-6vsh7" event={"ID":"b6ddfc52-3897-42d6-87cd-f78774091ecb","Type":"ContainerStarted","Data":"e243e8fa863aaa8b744df378514a3e80dc5fbb64983f07485a39483992a9c61d"} Aug 04 12:28:33.739290 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.739268 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" event={"ID":"9146f37a-339d-4372-a1f0-4b32dcee4a10","Type":"ContainerStarted","Data":"10d3b4498d4269c6c20589a622c6e2518a457a8656e492202f14f1a6006545a8"} Aug 04 12:28:33.775845 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.774353 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-jchjx" podStartSLOduration=62.031193929 podStartE2EDuration="1m6.774332457s" totalImagesPullingTime="4.743138528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:33.773633005 +0000 UTC m=+74.010064376" watchObservedRunningTime="2026-08-04 12:28:33.774332457 +0000 UTC m=+74.010763828" Aug 04 12:28:33.775845 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.774746 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-bfzn7" podStartSLOduration=34.234493383 podStartE2EDuration="37.774736162s" totalImagesPullingTime="3.540242779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:33.748618564 +0000 UTC m=+73.985049933" watchObservedRunningTime="2026-08-04 12:28:33.774736162 +0000 UTC m=+74.011167533" Aug 04 12:28:33.795998 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.795546 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tztk9" podStartSLOduration=67.401022265 podStartE2EDuration="1m13.795525646s" totalImagesPullingTime="6.394503381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:33.794584343 +0000 UTC m=+74.031015714" watchObservedRunningTime="2026-08-04 12:28:33.795525646 +0000 UTC m=+74.031957015" Aug 04 12:28:33.820245 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.820114 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jvvkp" podStartSLOduration=61.992634397 podStartE2EDuration="1m6.820096688s" totalImagesPullingTime="4.827462291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:33.818641104 +0000 UTC m=+74.055072474" watchObservedRunningTime="2026-08-04 12:28:33.820096688 +0000 UTC m=+74.056528058" Aug 04 12:28:33.851205 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.851151 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sj8xn" podStartSLOduration=33.989564125 podStartE2EDuration="38.851129435s" totalImagesPullingTime="4.86156531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:33.849562757 +0000 UTC m=+74.085994192" watchObservedRunningTime="2026-08-04 12:28:33.851129435 +0000 UTC m=+74.087560811" Aug 04 12:28:33.874249 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:33.874190 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr"] Aug 04 12:28:33.876840 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:33.876797 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb124e3c_963d_4201_be69_3ecb41e9a37e.slice/crio-f8c8004fbfe74ead2c1006c1b61302b89a097c61e1d55dca23292426ff8365b6 WatchSource:0}: Error finding container f8c8004fbfe74ead2c1006c1b61302b89a097c61e1d55dca23292426ff8365b6: Status 404 returned error can't find the container with id f8c8004fbfe74ead2c1006c1b61302b89a097c61e1d55dca23292426ff8365b6 Aug 04 12:28:34.747475 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:34.746877 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" event={"ID":"cb124e3c-963d-4201-be69-3ecb41e9a37e","Type":"ContainerStarted","Data":"f8c8004fbfe74ead2c1006c1b61302b89a097c61e1d55dca23292426ff8365b6"} Aug 04 12:28:36.759193 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:36.759096 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" event={"ID":"cb124e3c-963d-4201-be69-3ecb41e9a37e","Type":"ContainerStarted","Data":"055aebbe70f7ca0ad249526ae6524e0a7255d423df6a589c973ee085953b97ad"} Aug 04 12:28:36.759600 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:36.759257 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" Aug 04 12:28:36.764699 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:36.764679 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" Aug 04 12:28:36.777681 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:36.777634 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-w6dfr" podStartSLOduration=1.1669736290000001 podStartE2EDuration="3.777614917s" totalImagesPullingTime="2.610641288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:36.776486452 +0000 UTC m=+77.012917823" watchObservedRunningTime="2026-08-04 12:28:36.777614917 +0000 UTC m=+77.014046289" Aug 04 12:28:38.772358 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:38.772276 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" event={"ID":"9146f37a-339d-4372-a1f0-4b32dcee4a10","Type":"ContainerStarted","Data":"21ac3f4f78884f244695e89cfd82484039c3610b579e4ba8f327487d69da5ced"} Aug 04 12:28:38.772358 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:38.772347 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:38.774983 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:38.774923 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" Aug 04 12:28:38.795245 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:38.795193 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7795b7d46-v868b" podStartSLOduration=4.030747613 podStartE2EDuration="9.795175167s" totalImagesPullingTime="5.764427554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:38.794211538 +0000 UTC m=+79.030642907" watchObservedRunningTime="2026-08-04 12:28:38.795175167 +0000 UTC m=+79.031606537" Aug 04 12:28:44.755198 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.755070 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sj8xn" Aug 04 12:28:44.902933 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.902906 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hq7n4"] Aug 04 12:28:44.906242 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.906219 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.909932 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.909905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 12:28:44.910066 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.910044 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 12:28:44.910340 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.910321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gjl6d\"" Aug 04 12:28:44.910984 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.910941 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 12:28:44.911566 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.911547 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 12:28:44.927127 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-tls\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.927250 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-wtmp\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.927250 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927220 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.927250 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927247 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.927414 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-textfile\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.927414 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sr4b\" (UniqueName: \"kubernetes.io/projected/35ca5d62-6868-4709-92db-1121212793fa-kube-api-access-9sr4b\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.927414 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ca5d62-6868-4709-92db-1121212793fa-metrics-client-ca\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.927559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-root\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:44.927559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:44.927481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-sys\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028228 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-wtmp\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028228 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028443 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028443 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-textfile\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028443 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028352 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-wtmp\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028586 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sr4b\" (UniqueName: \"kubernetes.io/projected/35ca5d62-6868-4709-92db-1121212793fa-kube-api-access-9sr4b\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028586 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ca5d62-6868-4709-92db-1121212793fa-metrics-client-ca\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028586 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-root\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028701 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-sys\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028701 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-tls\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028701 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028637 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-textfile\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028862 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028702 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-sys\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028862 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028728 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35ca5d62-6868-4709-92db-1121212793fa-root\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028972 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:45.028913 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 04 12:28:45.028972 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.028944 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.028972 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:28:45.028971 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-tls podName:35ca5d62-6868-4709-92db-1121212793fa nodeName:}" failed. No retries permitted until 2026-08-04 12:28:45.528953169 +0000 UTC m=+85.765384517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-tls") pod "node-exporter-hq7n4" (UID: "35ca5d62-6868-4709-92db-1121212793fa") : secret "node-exporter-tls" not found Aug 04 12:28:45.029296 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.029274 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ca5d62-6868-4709-92db-1121212793fa-metrics-client-ca\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.031156 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.031121 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.042389 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.042370 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sr4b\" (UniqueName: \"kubernetes.io/projected/35ca5d62-6868-4709-92db-1121212793fa-kube-api-access-9sr4b\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.533315 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.533275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-tls\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.536078 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.536046 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35ca5d62-6868-4709-92db-1121212793fa-node-exporter-tls\") pod \"node-exporter-hq7n4\" (UID: \"35ca5d62-6868-4709-92db-1121212793fa\") " pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:45.818317 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:45.818285 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hq7n4" Aug 04 12:28:47.252723 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:47.252690 2580 patch_prober.go:28] interesting pod/image-registry-b8f7cc49d-hpblp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 12:28:47.253224 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:47.252752 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" podUID="1b579baf-6a8c-441e-af03-1a66c1e1457c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 12:28:49.712979 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:49.712947 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:28:50.003781 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:28:50.003750 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ca5d62_6868_4709_92db_1121212793fa.slice/crio-ed0ee8a2db51c53d80c41b4323a5a2b457f3a39fa6b728fbf84417fef64e15a1 WatchSource:0}: Error finding container ed0ee8a2db51c53d80c41b4323a5a2b457f3a39fa6b728fbf84417fef64e15a1: Status 404 returned error can't find the container with id ed0ee8a2db51c53d80c41b4323a5a2b457f3a39fa6b728fbf84417fef64e15a1 Aug 04 12:28:50.828256 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:50.827793 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-6vsh7" event={"ID":"b6ddfc52-3897-42d6-87cd-f78774091ecb","Type":"ContainerStarted","Data":"6388de9660b8311ceaaaba13570a2bd782eb31586bb374fbb3703012f6aa938e"} Aug 04 12:28:50.828256 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:50.827881 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-6vsh7" Aug 04 12:28:50.829677 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:50.829632 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hq7n4" event={"ID":"35ca5d62-6868-4709-92db-1121212793fa","Type":"ContainerStarted","Data":"ed0ee8a2db51c53d80c41b4323a5a2b457f3a39fa6b728fbf84417fef64e15a1"} Aug 04 12:28:50.844378 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:50.844343 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-6vsh7" Aug 04 12:28:50.851065 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:50.849207 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-6vsh7" podStartSLOduration=4.472446743 podStartE2EDuration="21.849188518s" totalImagesPullingTime="17.376741775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:50.84811758 +0000 UTC m=+91.084548953" watchObservedRunningTime="2026-08-04 12:28:50.849188518 +0000 UTC m=+91.085619889" Aug 04 12:28:51.836971 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:51.836933 2580 generic.go:356] "Generic (PLEG): container finished" podID="35ca5d62-6868-4709-92db-1121212793fa" containerID="bbf9f6f6433dad927099142bbd8cd695051ca1bacf3a400c49ce745b17a4a97d" exitCode=0 Aug 04 12:28:51.837459 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:51.837022 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hq7n4" event={"ID":"35ca5d62-6868-4709-92db-1121212793fa","Type":"ContainerDied","Data":"bbf9f6f6433dad927099142bbd8cd695051ca1bacf3a400c49ce745b17a4a97d"} Aug 04 12:28:52.469273 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:52.469236 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b8f7cc49d-hpblp"] Aug 04 12:28:52.844622 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:52.844587 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hq7n4" event={"ID":"35ca5d62-6868-4709-92db-1121212793fa","Type":"ContainerStarted","Data":"dc79d1d7a06771067b0576339c0b6f49ec25f71330d0fa5020fbd6b5c6a63001"} Aug 04 12:28:52.845081 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:52.844650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hq7n4" event={"ID":"35ca5d62-6868-4709-92db-1121212793fa","Type":"ContainerStarted","Data":"42956b034e92b17cd447d86e8df91d72b68fb7182936e001bfa9faa244924037"} Aug 04 12:28:52.871038 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:52.870989 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hq7n4" podStartSLOduration=7.854359878 podStartE2EDuration="8.870967604s" totalImagesPullingTime="929.319975ms" totalInitContainerRuntime="87.287751ms" isStatefulPod=true podCreationTimestamp="2026-08-04 12:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:52.869366746 +0000 UTC m=+93.105798116" watchObservedRunningTime="2026-08-04 12:28:52.870967604 +0000 UTC m=+93.107398974" Aug 04 12:28:57.681554 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:28:57.681516 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2hvfq" Aug 04 12:29:16.937302 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:16.937269 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b4ada05-f9a1-4310-aee9-169f9584c2dc" containerID="25ac65e80c836a3e727262ba0c4ef6c8f9c917c3917faed0d54e64a984b04387" exitCode=0 Aug 04 12:29:16.937696 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:16.937343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" event={"ID":"5b4ada05-f9a1-4310-aee9-169f9584c2dc","Type":"ContainerDied","Data":"25ac65e80c836a3e727262ba0c4ef6c8f9c917c3917faed0d54e64a984b04387"} Aug 04 12:29:16.937696 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:16.937656 2580 scope.go:117] "RemoveContainer" containerID="25ac65e80c836a3e727262ba0c4ef6c8f9c917c3917faed0d54e64a984b04387" Aug 04 12:29:17.494009 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.493943 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" podUID="1b579baf-6a8c-441e-af03-1a66c1e1457c" containerName="registry" containerID="cri-o://8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623" gracePeriod=30 Aug 04 12:29:17.785714 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.785686 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:29:17.914659 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.914629 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-installation-pull-secrets\") pod \"1b579baf-6a8c-441e-af03-1a66c1e1457c\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " Aug 04 12:29:17.914659 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.914664 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-certificates\") pod \"1b579baf-6a8c-441e-af03-1a66c1e1457c\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " Aug 04 12:29:17.914942 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.914686 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdlk9\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-kube-api-access-sdlk9\") pod \"1b579baf-6a8c-441e-af03-1a66c1e1457c\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " Aug 04 12:29:17.914942 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.914729 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") pod \"1b579baf-6a8c-441e-af03-1a66c1e1457c\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " Aug 04 12:29:17.914942 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.914762 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b579baf-6a8c-441e-af03-1a66c1e1457c-ca-trust-extracted\") pod \"1b579baf-6a8c-441e-af03-1a66c1e1457c\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " Aug 04 12:29:17.914942 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.914796 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-bound-sa-token\") pod \"1b579baf-6a8c-441e-af03-1a66c1e1457c\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " Aug 04 12:29:17.914942 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.914891 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-trusted-ca\") pod \"1b579baf-6a8c-441e-af03-1a66c1e1457c\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " Aug 04 12:29:17.914942 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.914922 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-image-registry-private-configuration\") pod \"1b579baf-6a8c-441e-af03-1a66c1e1457c\" (UID: \"1b579baf-6a8c-441e-af03-1a66c1e1457c\") " Aug 04 12:29:17.915255 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.915181 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1b579baf-6a8c-441e-af03-1a66c1e1457c" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:29:17.915746 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.915682 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1b579baf-6a8c-441e-af03-1a66c1e1457c" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:29:17.917362 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.917326 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1b579baf-6a8c-441e-af03-1a66c1e1457c" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 12:29:17.917503 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.917477 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1b579baf-6a8c-441e-af03-1a66c1e1457c" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 12:29:17.917573 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.917542 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-kube-api-access-sdlk9" (OuterVolumeSpecName: "kube-api-access-sdlk9") pod "1b579baf-6a8c-441e-af03-1a66c1e1457c" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c"). InnerVolumeSpecName "kube-api-access-sdlk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:29:17.917724 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.917703 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1b579baf-6a8c-441e-af03-1a66c1e1457c" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:29:17.917771 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.917747 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1b579baf-6a8c-441e-af03-1a66c1e1457c" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:29:17.923970 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.923943 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b579baf-6a8c-441e-af03-1a66c1e1457c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1b579baf-6a8c-441e-af03-1a66c1e1457c" (UID: "1b579baf-6a8c-441e-af03-1a66c1e1457c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:29:17.942761 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.942733 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-xr9h6" event={"ID":"5b4ada05-f9a1-4310-aee9-169f9584c2dc","Type":"ContainerStarted","Data":"ad840a874b1fae211c014ac975248a149843d726458714b3c0c0e466e5ea3ed0"} Aug 04 12:29:17.943959 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.943941 2580 generic.go:356] "Generic (PLEG): container finished" podID="1b579baf-6a8c-441e-af03-1a66c1e1457c" containerID="8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623" exitCode=0 Aug 04 12:29:17.944019 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.943976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" event={"ID":"1b579baf-6a8c-441e-af03-1a66c1e1457c","Type":"ContainerDied","Data":"8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623"} Aug 04 12:29:17.944019 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.943982 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" Aug 04 12:29:17.944019 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.944007 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8f7cc49d-hpblp" event={"ID":"1b579baf-6a8c-441e-af03-1a66c1e1457c","Type":"ContainerDied","Data":"2e0bcc4a6480093744f1198612797c30be9c5c6d7afc93fc96ae8f1af2babb4b"} Aug 04 12:29:17.944114 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.944029 2580 scope.go:117] "RemoveContainer" containerID="8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623" Aug 04 12:29:17.952736 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.952723 2580 scope.go:117] "RemoveContainer" containerID="8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623" Aug 04 12:29:17.953030 ip-10-0-132-96 kubenswrapper[2580]: E0804 12:29:17.953006 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623\": container with ID starting with 8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623 not found: ID does not exist" containerID="8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623" Aug 04 12:29:17.953114 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.953040 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623"} err="failed to get container status \"8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623\": rpc error: code = NotFound desc = could not find container \"8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623\": container with ID starting with 8a548f7fafbc7d49ee27b42c07d85230d81dd3a1f4d1d50fdae99b24924be623 not found: ID does not exist" Aug 04 12:29:17.983711 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.983681 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b8f7cc49d-hpblp"] Aug 04 12:29:17.992182 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:17.992154 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b8f7cc49d-hpblp"] Aug 04 12:29:18.016536 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.016472 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-trusted-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:29:18.016536 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.016499 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-image-registry-private-configuration\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:29:18.016536 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.016516 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b579baf-6a8c-441e-af03-1a66c1e1457c-installation-pull-secrets\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:29:18.016536 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.016532 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-certificates\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:29:18.016731 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.016547 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdlk9\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-kube-api-access-sdlk9\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:29:18.016731 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.016558 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-registry-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:29:18.016731 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.016566 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b579baf-6a8c-441e-af03-1a66c1e1457c-ca-trust-extracted\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:29:18.016731 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.016574 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b579baf-6a8c-441e-af03-1a66c1e1457c-bound-sa-token\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:29:18.276235 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:18.276154 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b579baf-6a8c-441e-af03-1a66c1e1457c" path="/var/lib/kubelet/pods/1b579baf-6a8c-441e-af03-1a66c1e1457c/volumes" Aug 04 12:29:25.973110 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:25.973066 2580 generic.go:356] "Generic (PLEG): container finished" podID="6a350f36-d1a5-462b-8f92-eea0d0a47586" containerID="c3b6c3c6ec21aa2d15eb333a1eb6b0c7ff3b5b36933160d39bbb29ffd195d755" exitCode=0 Aug 04 12:29:25.973547 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:25.973136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-j56ss" event={"ID":"6a350f36-d1a5-462b-8f92-eea0d0a47586","Type":"ContainerDied","Data":"c3b6c3c6ec21aa2d15eb333a1eb6b0c7ff3b5b36933160d39bbb29ffd195d755"} Aug 04 12:29:25.973547 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:25.973504 2580 scope.go:117] "RemoveContainer" containerID="c3b6c3c6ec21aa2d15eb333a1eb6b0c7ff3b5b36933160d39bbb29ffd195d755" Aug 04 12:29:26.978899 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:26.978861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-j56ss" event={"ID":"6a350f36-d1a5-462b-8f92-eea0d0a47586","Type":"ContainerStarted","Data":"69782872435acc3cbac0e09431bf33bdcd985ec9d768f5a48b9d6ff8f299cd5a"} Aug 04 12:29:37.015102 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:37.015066 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f297761-617d-489b-b1f1-710bc3d8b11c" containerID="066aac4155c398b4834644ec81976e12fb390111f93e67f578a97d6fe421e895" exitCode=0 Aug 04 12:29:37.015520 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:37.015108 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" event={"ID":"0f297761-617d-489b-b1f1-710bc3d8b11c","Type":"ContainerDied","Data":"066aac4155c398b4834644ec81976e12fb390111f93e67f578a97d6fe421e895"} Aug 04 12:29:37.015520 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:37.015443 2580 scope.go:117] "RemoveContainer" containerID="066aac4155c398b4834644ec81976e12fb390111f93e67f578a97d6fe421e895" Aug 04 12:29:38.021533 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:29:38.021496 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-f2k9x" event={"ID":"0f297761-617d-489b-b1f1-710bc3d8b11c","Type":"ContainerStarted","Data":"3e60b3e4c8e6a5e2779757c4b63dab49e43e95a708c87df4fd80162a82c34ce9"} Aug 04 12:30:09.460688 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.460654 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6766476b6b-s744z"] Aug 04 12:30:09.461178 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.461131 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b579baf-6a8c-441e-af03-1a66c1e1457c" containerName="registry" Aug 04 12:30:09.461178 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.461149 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b579baf-6a8c-441e-af03-1a66c1e1457c" containerName="registry" Aug 04 12:30:09.461278 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.461217 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b579baf-6a8c-441e-af03-1a66c1e1457c" containerName="registry" Aug 04 12:30:09.464425 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.464408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.467149 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.467126 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Aug 04 12:30:09.467264 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.467150 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Aug 04 12:30:09.467264 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.467126 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Aug 04 12:30:09.467813 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.467794 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Aug 04 12:30:09.468578 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.468553 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Aug 04 12:30:09.468706 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.468553 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5w8dn\"" Aug 04 12:30:09.489158 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.489124 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6766476b6b-s744z"] Aug 04 12:30:09.490183 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.490157 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Aug 04 12:30:09.615074 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.615036 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-federate-client-tls\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.615263 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.615110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.615263 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.615170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.615263 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.615218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-metrics-client-ca\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.615263 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.615248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-secret-telemeter-client\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.615413 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.615269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chpz2\" (UniqueName: \"kubernetes.io/projected/51caa8d7-0b5b-413a-ad24-214e17fdcd88-kube-api-access-chpz2\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.615413 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.615325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-serving-certs-ca-bundle\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.615413 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.615358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-telemeter-client-tls\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.716624 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.716538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.716624 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.716576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-metrics-client-ca\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.716624 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.716596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-secret-telemeter-client\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.716624 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.716613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chpz2\" (UniqueName: \"kubernetes.io/projected/51caa8d7-0b5b-413a-ad24-214e17fdcd88-kube-api-access-chpz2\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.716993 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.716641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-serving-certs-ca-bundle\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.716993 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.716669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-telemeter-client-tls\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.716993 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.716717 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-federate-client-tls\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.716993 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.716763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.717435 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.717410 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-metrics-client-ca\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.717539 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.717468 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-serving-certs-ca-bundle\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.717734 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.717710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51caa8d7-0b5b-413a-ad24-214e17fdcd88-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.719447 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.719418 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-telemeter-client-tls\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.719553 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.719419 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-federate-client-tls\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.719894 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.719873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.720006 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.719987 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/51caa8d7-0b5b-413a-ad24-214e17fdcd88-secret-telemeter-client\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.731073 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.731049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chpz2\" (UniqueName: \"kubernetes.io/projected/51caa8d7-0b5b-413a-ad24-214e17fdcd88-kube-api-access-chpz2\") pod \"telemeter-client-6766476b6b-s744z\" (UID: \"51caa8d7-0b5b-413a-ad24-214e17fdcd88\") " pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.775119 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.775083 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" Aug 04 12:30:09.917684 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:09.917657 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6766476b6b-s744z"] Aug 04 12:30:09.920341 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:30:09.920312 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51caa8d7_0b5b_413a_ad24_214e17fdcd88.slice/crio-9921cead4b07bba57c0b2544a6878790f1563f70d02be9580f5cea57c1ebf26f WatchSource:0}: Error finding container 9921cead4b07bba57c0b2544a6878790f1563f70d02be9580f5cea57c1ebf26f: Status 404 returned error can't find the container with id 9921cead4b07bba57c0b2544a6878790f1563f70d02be9580f5cea57c1ebf26f Aug 04 12:30:10.142798 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:10.142765 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" event={"ID":"51caa8d7-0b5b-413a-ad24-214e17fdcd88","Type":"ContainerStarted","Data":"9921cead4b07bba57c0b2544a6878790f1563f70d02be9580f5cea57c1ebf26f"} Aug 04 12:30:12.153006 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:12.152971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" event={"ID":"51caa8d7-0b5b-413a-ad24-214e17fdcd88","Type":"ContainerStarted","Data":"d7e9b736814214fd081a0b254e87ed6929add1d01aa4dc7b6677af4a82367aa1"} Aug 04 12:30:14.165348 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:14.165310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" event={"ID":"51caa8d7-0b5b-413a-ad24-214e17fdcd88","Type":"ContainerStarted","Data":"fe74adfd6223720bac92f2348bfd17da5a6c8fb561f6efeae9767d8b6fb9b772"} Aug 04 12:30:14.165798 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:14.165399 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" event={"ID":"51caa8d7-0b5b-413a-ad24-214e17fdcd88","Type":"ContainerStarted","Data":"bce8a74e4853f21616c1bf56871a50767a050fae33a16a5934b1aaa3839f73c9"} Aug 04 12:30:14.193268 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:30:14.192963 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6766476b6b-s744z" podStartSLOduration=1.402496311 podStartE2EDuration="5.192935776s" totalImagesPullingTime="3.790439465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:30:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:30:14.19151798 +0000 UTC m=+174.427949343" watchObservedRunningTime="2026-08-04 12:30:14.192935776 +0000 UTC m=+174.429367147" Aug 04 12:32:20.179739 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:20.179711 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:32:20.183391 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:20.183370 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:32:20.189759 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:20.189735 2580 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 12:32:52.457553 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.457518 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv"] Aug 04 12:32:52.460725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.460710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.463398 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.463375 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 12:32:52.464501 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.464486 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 12:32:52.464564 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.464520 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2n8x9\"" Aug 04 12:32:52.471615 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.471589 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv"] Aug 04 12:32:52.613649 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.613603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqjb4\" (UniqueName: \"kubernetes.io/projected/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-kube-api-access-bqjb4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.613870 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.613668 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.613870 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.613702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.715118 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.715052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.715118 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.715091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.715284 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.715144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqjb4\" (UniqueName: \"kubernetes.io/projected/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-kube-api-access-bqjb4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.715434 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.715415 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.715504 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.715485 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.725905 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.725879 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqjb4\" (UniqueName: \"kubernetes.io/projected/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-kube-api-access-bqjb4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.770035 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.770009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:32:52.900800 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.900752 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv"] Aug 04 12:32:52.903214 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:32:52.903186 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a0b40b_9057_4401_aa8a_23d7f11dd27d.slice/crio-cf8b0e598568967e3daa54c5c93b07789ba6ceda51a14c5b6c179955c442acd5 WatchSource:0}: Error finding container cf8b0e598568967e3daa54c5c93b07789ba6ceda51a14c5b6c179955c442acd5: Status 404 returned error can't find the container with id cf8b0e598568967e3daa54c5c93b07789ba6ceda51a14c5b6c179955c442acd5 Aug 04 12:32:52.905162 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:52.905144 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 12:32:53.765662 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:53.765626 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" event={"ID":"a7a0b40b-9057-4401-aa8a-23d7f11dd27d","Type":"ContainerStarted","Data":"cf8b0e598568967e3daa54c5c93b07789ba6ceda51a14c5b6c179955c442acd5"} Aug 04 12:32:59.793052 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:59.793016 2580 generic.go:356] "Generic (PLEG): container finished" podID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerID="7082608f84902b9101c00e04b192732fbe911346efe4375fe06c553d9b2006d6" exitCode=0 Aug 04 12:32:59.793522 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:32:59.793093 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" event={"ID":"a7a0b40b-9057-4401-aa8a-23d7f11dd27d","Type":"ContainerDied","Data":"7082608f84902b9101c00e04b192732fbe911346efe4375fe06c553d9b2006d6"} Aug 04 12:33:02.808365 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:02.808331 2580 generic.go:356] "Generic (PLEG): container finished" podID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerID="76849e3ff23178fea79cc1186f1c72f305d6268efb2b60a427830059462d7f75" exitCode=0 Aug 04 12:33:02.808720 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:02.808405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" event={"ID":"a7a0b40b-9057-4401-aa8a-23d7f11dd27d","Type":"ContainerDied","Data":"76849e3ff23178fea79cc1186f1c72f305d6268efb2b60a427830059462d7f75"} Aug 04 12:33:10.843312 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:10.843276 2580 generic.go:356] "Generic (PLEG): container finished" podID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerID="e6aec4fcb9a48cb73ec90ff776e19077a4a978563e60069059d9f31087daf36e" exitCode=0 Aug 04 12:33:10.843690 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:10.843363 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" event={"ID":"a7a0b40b-9057-4401-aa8a-23d7f11dd27d","Type":"ContainerDied","Data":"e6aec4fcb9a48cb73ec90ff776e19077a4a978563e60069059d9f31087daf36e"} Aug 04 12:33:11.973157 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:11.973131 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:33:12.077022 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.076986 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-util\") pod \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " Aug 04 12:33:12.077192 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.077048 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqjb4\" (UniqueName: \"kubernetes.io/projected/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-kube-api-access-bqjb4\") pod \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " Aug 04 12:33:12.077192 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.077126 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-bundle\") pod \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\" (UID: \"a7a0b40b-9057-4401-aa8a-23d7f11dd27d\") " Aug 04 12:33:12.077720 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.077692 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-bundle" (OuterVolumeSpecName: "bundle") pod "a7a0b40b-9057-4401-aa8a-23d7f11dd27d" (UID: "a7a0b40b-9057-4401-aa8a-23d7f11dd27d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:33:12.079574 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.079553 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-kube-api-access-bqjb4" (OuterVolumeSpecName: "kube-api-access-bqjb4") pod "a7a0b40b-9057-4401-aa8a-23d7f11dd27d" (UID: "a7a0b40b-9057-4401-aa8a-23d7f11dd27d"). InnerVolumeSpecName "kube-api-access-bqjb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:33:12.081094 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.081069 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-util" (OuterVolumeSpecName: "util") pod "a7a0b40b-9057-4401-aa8a-23d7f11dd27d" (UID: "a7a0b40b-9057-4401-aa8a-23d7f11dd27d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:33:12.178023 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.177936 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-util\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:12.178023 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.177967 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqjb4\" (UniqueName: \"kubernetes.io/projected/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-kube-api-access-bqjb4\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:12.178023 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.177976 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a0b40b-9057-4401-aa8a-23d7f11dd27d-bundle\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:12.853367 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.853330 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" event={"ID":"a7a0b40b-9057-4401-aa8a-23d7f11dd27d","Type":"ContainerDied","Data":"cf8b0e598568967e3daa54c5c93b07789ba6ceda51a14c5b6c179955c442acd5"} Aug 04 12:33:12.853367 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.853354 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk4rv" Aug 04 12:33:12.853367 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:12.853376 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf8b0e598568967e3daa54c5c93b07789ba6ceda51a14c5b6c179955c442acd5" Aug 04 12:33:24.863345 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.863314 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c"] Aug 04 12:33:24.863792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.863615 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerName="util" Aug 04 12:33:24.863792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.863625 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerName="util" Aug 04 12:33:24.863792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.863636 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerName="extract" Aug 04 12:33:24.863792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.863641 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerName="extract" Aug 04 12:33:24.863792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.863649 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerName="pull" Aug 04 12:33:24.863792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.863654 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerName="pull" Aug 04 12:33:24.863792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.863698 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7a0b40b-9057-4401-aa8a-23d7f11dd27d" containerName="extract" Aug 04 12:33:24.867041 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.867022 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:24.869709 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.869688 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 12:33:24.869883 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.869865 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 12:33:24.870907 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.870889 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2n8x9\"" Aug 04 12:33:24.882126 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.882103 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c"] Aug 04 12:33:24.983041 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.983006 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhnhh\" (UniqueName: \"kubernetes.io/projected/181ad7f5-674c-41de-840e-875ea56ce683-kube-api-access-bhnhh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:24.983220 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.983052 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:24.983220 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:24.983072 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:25.084390 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.084355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhnhh\" (UniqueName: \"kubernetes.io/projected/181ad7f5-674c-41de-840e-875ea56ce683-kube-api-access-bhnhh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:25.084489 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.084417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:25.084489 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.084454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:25.084771 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.084750 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:25.084820 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.084803 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:25.095792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.095775 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhnhh\" (UniqueName: \"kubernetes.io/projected/181ad7f5-674c-41de-840e-875ea56ce683-kube-api-access-bhnhh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:25.176118 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.176042 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:25.311670 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.311648 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c"] Aug 04 12:33:25.313868 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:33:25.313811 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181ad7f5_674c_41de_840e_875ea56ce683.slice/crio-c58e6d982ca1d46773d1a1b4bb072baaf4558745d0afeae8f41799e1dbc09b43 WatchSource:0}: Error finding container c58e6d982ca1d46773d1a1b4bb072baaf4558745d0afeae8f41799e1dbc09b43: Status 404 returned error can't find the container with id c58e6d982ca1d46773d1a1b4bb072baaf4558745d0afeae8f41799e1dbc09b43 Aug 04 12:33:25.904194 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.904160 2580 generic.go:356] "Generic (PLEG): container finished" podID="181ad7f5-674c-41de-840e-875ea56ce683" containerID="10ac85f8e38d47441776927d733ad71b54195037befa252a6a5c0372dd070520" exitCode=0 Aug 04 12:33:25.904573 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.904248 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" event={"ID":"181ad7f5-674c-41de-840e-875ea56ce683","Type":"ContainerDied","Data":"10ac85f8e38d47441776927d733ad71b54195037befa252a6a5c0372dd070520"} Aug 04 12:33:25.904573 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:25.904301 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" event={"ID":"181ad7f5-674c-41de-840e-875ea56ce683","Type":"ContainerStarted","Data":"c58e6d982ca1d46773d1a1b4bb072baaf4558745d0afeae8f41799e1dbc09b43"} Aug 04 12:33:27.143562 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.143534 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jlkq5"] Aug 04 12:33:27.145918 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.145899 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:27.148524 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.148499 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 04 12:33:27.148652 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.148574 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 04 12:33:27.148787 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.148772 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-xbn74\"" Aug 04 12:33:27.156603 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.156579 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jlkq5"] Aug 04 12:33:27.202208 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.202173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j6kr\" (UniqueName: \"kubernetes.io/projected/fe890b32-92ee-469e-aa96-4a486bd7cefd-kube-api-access-6j6kr\") pod \"cert-manager-webhook-6486ffd997-jlkq5\" (UID: \"fe890b32-92ee-469e-aa96-4a486bd7cefd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:27.202400 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.202231 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe890b32-92ee-469e-aa96-4a486bd7cefd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jlkq5\" (UID: \"fe890b32-92ee-469e-aa96-4a486bd7cefd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:27.302739 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.302707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j6kr\" (UniqueName: \"kubernetes.io/projected/fe890b32-92ee-469e-aa96-4a486bd7cefd-kube-api-access-6j6kr\") pod \"cert-manager-webhook-6486ffd997-jlkq5\" (UID: \"fe890b32-92ee-469e-aa96-4a486bd7cefd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:27.302934 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.302780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe890b32-92ee-469e-aa96-4a486bd7cefd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jlkq5\" (UID: \"fe890b32-92ee-469e-aa96-4a486bd7cefd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:27.312761 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.312727 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe890b32-92ee-469e-aa96-4a486bd7cefd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jlkq5\" (UID: \"fe890b32-92ee-469e-aa96-4a486bd7cefd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:27.312914 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.312781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j6kr\" (UniqueName: \"kubernetes.io/projected/fe890b32-92ee-469e-aa96-4a486bd7cefd-kube-api-access-6j6kr\") pod \"cert-manager-webhook-6486ffd997-jlkq5\" (UID: \"fe890b32-92ee-469e-aa96-4a486bd7cefd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:27.463379 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.463292 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:27.612158 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.612134 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jlkq5"] Aug 04 12:33:27.614309 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:33:27.614274 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe890b32_92ee_469e_aa96_4a486bd7cefd.slice/crio-b054c58424f57b07191fb83e1ce166c4039f2ee016cf2bb2ed4ee1160f6aeddc WatchSource:0}: Error finding container b054c58424f57b07191fb83e1ce166c4039f2ee016cf2bb2ed4ee1160f6aeddc: Status 404 returned error can't find the container with id b054c58424f57b07191fb83e1ce166c4039f2ee016cf2bb2ed4ee1160f6aeddc Aug 04 12:33:27.914077 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:27.914040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" event={"ID":"fe890b32-92ee-469e-aa96-4a486bd7cefd","Type":"ContainerStarted","Data":"b054c58424f57b07191fb83e1ce166c4039f2ee016cf2bb2ed4ee1160f6aeddc"} Aug 04 12:33:28.920638 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:28.920597 2580 generic.go:356] "Generic (PLEG): container finished" podID="181ad7f5-674c-41de-840e-875ea56ce683" containerID="6ede8431e121746491086c5f7374f611b00eee2e75e3c197cec5512b220988ba" exitCode=0 Aug 04 12:33:28.921126 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:28.920719 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" event={"ID":"181ad7f5-674c-41de-840e-875ea56ce683","Type":"ContainerDied","Data":"6ede8431e121746491086c5f7374f611b00eee2e75e3c197cec5512b220988ba"} Aug 04 12:33:29.928327 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:29.928297 2580 generic.go:356] "Generic (PLEG): container finished" podID="181ad7f5-674c-41de-840e-875ea56ce683" containerID="90842baafd26362867b1eb49e9a526329e6113e1073c584ba9d02fd2bfa8f82a" exitCode=0 Aug 04 12:33:29.928754 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:29.928377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" event={"ID":"181ad7f5-674c-41de-840e-875ea56ce683","Type":"ContainerDied","Data":"90842baafd26362867b1eb49e9a526329e6113e1073c584ba9d02fd2bfa8f82a"} Aug 04 12:33:30.934088 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:30.934055 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" event={"ID":"fe890b32-92ee-469e-aa96-4a486bd7cefd","Type":"ContainerStarted","Data":"493954ce14d7ba1a6c258d6702afb2fcc33894874d6a79a9f82a97bddbdac1ec"} Aug 04 12:33:30.934530 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:30.934209 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:30.953531 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:30.953484 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" podStartSLOduration=1.225493889 podStartE2EDuration="3.95346215s" totalImagesPullingTime="2.727968261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:33:30.952340406 +0000 UTC m=+371.188771775" watchObservedRunningTime="2026-08-04 12:33:30.95346215 +0000 UTC m=+371.189893521" Aug 04 12:33:31.062746 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.062719 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:31.136614 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.136584 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhnhh\" (UniqueName: \"kubernetes.io/projected/181ad7f5-674c-41de-840e-875ea56ce683-kube-api-access-bhnhh\") pod \"181ad7f5-674c-41de-840e-875ea56ce683\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " Aug 04 12:33:31.136775 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.136628 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-util\") pod \"181ad7f5-674c-41de-840e-875ea56ce683\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " Aug 04 12:33:31.136775 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.136703 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-bundle\") pod \"181ad7f5-674c-41de-840e-875ea56ce683\" (UID: \"181ad7f5-674c-41de-840e-875ea56ce683\") " Aug 04 12:33:31.137186 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.137130 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-bundle" (OuterVolumeSpecName: "bundle") pod "181ad7f5-674c-41de-840e-875ea56ce683" (UID: "181ad7f5-674c-41de-840e-875ea56ce683"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:33:31.138851 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.138813 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181ad7f5-674c-41de-840e-875ea56ce683-kube-api-access-bhnhh" (OuterVolumeSpecName: "kube-api-access-bhnhh") pod "181ad7f5-674c-41de-840e-875ea56ce683" (UID: "181ad7f5-674c-41de-840e-875ea56ce683"). InnerVolumeSpecName "kube-api-access-bhnhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:33:31.141229 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.141203 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-util" (OuterVolumeSpecName: "util") pod "181ad7f5-674c-41de-840e-875ea56ce683" (UID: "181ad7f5-674c-41de-840e-875ea56ce683"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:33:31.238056 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.237958 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-bundle\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:31.238056 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.238001 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhnhh\" (UniqueName: \"kubernetes.io/projected/181ad7f5-674c-41de-840e-875ea56ce683-kube-api-access-bhnhh\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:31.238056 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.238012 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/181ad7f5-674c-41de-840e-875ea56ce683-util\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:31.940164 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.940128 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" Aug 04 12:33:31.940164 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.940139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4rp4c" event={"ID":"181ad7f5-674c-41de-840e-875ea56ce683","Type":"ContainerDied","Data":"c58e6d982ca1d46773d1a1b4bb072baaf4558745d0afeae8f41799e1dbc09b43"} Aug 04 12:33:31.940622 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:31.940189 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c58e6d982ca1d46773d1a1b4bb072baaf4558745d0afeae8f41799e1dbc09b43" Aug 04 12:33:36.943225 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:36.943196 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jlkq5" Aug 04 12:33:40.535322 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.535289 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc"] Aug 04 12:33:40.535704 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.535562 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181ad7f5-674c-41de-840e-875ea56ce683" containerName="pull" Aug 04 12:33:40.535704 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.535572 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="181ad7f5-674c-41de-840e-875ea56ce683" containerName="pull" Aug 04 12:33:40.535704 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.535585 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181ad7f5-674c-41de-840e-875ea56ce683" containerName="extract" Aug 04 12:33:40.535704 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.535590 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="181ad7f5-674c-41de-840e-875ea56ce683" containerName="extract" Aug 04 12:33:40.535704 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.535605 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181ad7f5-674c-41de-840e-875ea56ce683" containerName="util" Aug 04 12:33:40.535704 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.535611 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="181ad7f5-674c-41de-840e-875ea56ce683" containerName="util" Aug 04 12:33:40.535704 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.535656 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="181ad7f5-674c-41de-840e-875ea56ce683" containerName="extract" Aug 04 12:33:40.549242 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.549216 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc"] Aug 04 12:33:40.549365 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.549316 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" Aug 04 12:33:40.552183 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.552161 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 12:33:40.553199 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.553159 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 12:33:40.553328 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.553200 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lfqnc\"" Aug 04 12:33:40.714923 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.714888 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46008ad0-353e-4e70-87bc-a35a2f7c86ee-tmp\") pod \"openshift-lws-operator-fd8ccff4c-587zc\" (UID: \"46008ad0-353e-4e70-87bc-a35a2f7c86ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" Aug 04 12:33:40.714923 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.714921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bkhs\" (UniqueName: \"kubernetes.io/projected/46008ad0-353e-4e70-87bc-a35a2f7c86ee-kube-api-access-8bkhs\") pod \"openshift-lws-operator-fd8ccff4c-587zc\" (UID: \"46008ad0-353e-4e70-87bc-a35a2f7c86ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" Aug 04 12:33:40.816325 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.816299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46008ad0-353e-4e70-87bc-a35a2f7c86ee-tmp\") pod \"openshift-lws-operator-fd8ccff4c-587zc\" (UID: \"46008ad0-353e-4e70-87bc-a35a2f7c86ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" Aug 04 12:33:40.816502 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.816333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bkhs\" (UniqueName: \"kubernetes.io/projected/46008ad0-353e-4e70-87bc-a35a2f7c86ee-kube-api-access-8bkhs\") pod \"openshift-lws-operator-fd8ccff4c-587zc\" (UID: \"46008ad0-353e-4e70-87bc-a35a2f7c86ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" Aug 04 12:33:40.816684 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.816664 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46008ad0-353e-4e70-87bc-a35a2f7c86ee-tmp\") pod \"openshift-lws-operator-fd8ccff4c-587zc\" (UID: \"46008ad0-353e-4e70-87bc-a35a2f7c86ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" Aug 04 12:33:40.827147 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.827120 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bkhs\" (UniqueName: \"kubernetes.io/projected/46008ad0-353e-4e70-87bc-a35a2f7c86ee-kube-api-access-8bkhs\") pod \"openshift-lws-operator-fd8ccff4c-587zc\" (UID: \"46008ad0-353e-4e70-87bc-a35a2f7c86ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" Aug 04 12:33:40.859379 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.859354 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" Aug 04 12:33:40.986604 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:40.986582 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc"] Aug 04 12:33:40.989099 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:33:40.989071 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46008ad0_353e_4e70_87bc_a35a2f7c86ee.slice/crio-b54abe5ceca3d7577b6f1235a7ecef6785f02269b19d7fe044d27e0258a99277 WatchSource:0}: Error finding container b54abe5ceca3d7577b6f1235a7ecef6785f02269b19d7fe044d27e0258a99277: Status 404 returned error can't find the container with id b54abe5ceca3d7577b6f1235a7ecef6785f02269b19d7fe044d27e0258a99277 Aug 04 12:33:41.984670 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:41.984633 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" event={"ID":"46008ad0-353e-4e70-87bc-a35a2f7c86ee","Type":"ContainerStarted","Data":"b54abe5ceca3d7577b6f1235a7ecef6785f02269b19d7fe044d27e0258a99277"} Aug 04 12:33:43.995189 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:43.995151 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" event={"ID":"46008ad0-353e-4e70-87bc-a35a2f7c86ee","Type":"ContainerStarted","Data":"11eec197ebdee0b146cc10cb229543df32cb53606d560bcc6092a683f5046e59"} Aug 04 12:33:44.013791 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:44.013740 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587zc" podStartSLOduration=2.046481208 podStartE2EDuration="4.013726269s" totalImagesPullingTime="1.967245061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:33:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:33:44.013255422 +0000 UTC m=+384.249686792" watchObservedRunningTime="2026-08-04 12:33:44.013726269 +0000 UTC m=+384.250157640" Aug 04 12:33:46.588391 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.588360 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj"] Aug 04 12:33:46.591896 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.591881 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.594727 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.594705 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 12:33:46.594727 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.594729 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 12:33:46.595772 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.595757 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2n8x9\"" Aug 04 12:33:46.602217 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.602197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj"] Aug 04 12:33:46.764433 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.764399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4z2d\" (UniqueName: \"kubernetes.io/projected/efcbc421-f0f2-4038-ad0d-2a212d591cd9-kube-api-access-z4z2d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.764433 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.764431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.764650 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.764473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.865728 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.865652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.865728 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.865719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4z2d\" (UniqueName: \"kubernetes.io/projected/efcbc421-f0f2-4038-ad0d-2a212d591cd9-kube-api-access-z4z2d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.865925 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.865752 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.866128 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.866106 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.866165 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.866116 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.876715 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.876690 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4z2d\" (UniqueName: \"kubernetes.io/projected/efcbc421-f0f2-4038-ad0d-2a212d591cd9-kube-api-access-z4z2d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:46.901252 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:46.901230 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:47.043425 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:47.043402 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj"] Aug 04 12:33:47.044980 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:33:47.044951 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefcbc421_f0f2_4038_ad0d_2a212d591cd9.slice/crio-5f2961615e90127f6ec12563d23154f275abcea11521ece4f192bf60b690a9d6 WatchSource:0}: Error finding container 5f2961615e90127f6ec12563d23154f275abcea11521ece4f192bf60b690a9d6: Status 404 returned error can't find the container with id 5f2961615e90127f6ec12563d23154f275abcea11521ece4f192bf60b690a9d6 Aug 04 12:33:48.015582 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:48.015547 2580 generic.go:356] "Generic (PLEG): container finished" podID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerID="4a5adba49707557acc9677d0319847e324b53400431198f6ecca8f7d16065985" exitCode=0 Aug 04 12:33:48.015970 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:48.015594 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" event={"ID":"efcbc421-f0f2-4038-ad0d-2a212d591cd9","Type":"ContainerDied","Data":"4a5adba49707557acc9677d0319847e324b53400431198f6ecca8f7d16065985"} Aug 04 12:33:48.015970 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:48.015639 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" event={"ID":"efcbc421-f0f2-4038-ad0d-2a212d591cd9","Type":"ContainerStarted","Data":"5f2961615e90127f6ec12563d23154f275abcea11521ece4f192bf60b690a9d6"} Aug 04 12:33:49.022054 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:49.021964 2580 generic.go:356] "Generic (PLEG): container finished" podID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerID="6823068148ba955fd2e9ea4b4965323abd6c5a69eac1c5086dc561027e545388" exitCode=0 Aug 04 12:33:49.022054 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:49.022019 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" event={"ID":"efcbc421-f0f2-4038-ad0d-2a212d591cd9","Type":"ContainerDied","Data":"6823068148ba955fd2e9ea4b4965323abd6c5a69eac1c5086dc561027e545388"} Aug 04 12:33:50.027564 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:50.027531 2580 generic.go:356] "Generic (PLEG): container finished" podID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerID="4354f83cdc8998d4702aa1f12b3e3f3676db65466962fe03264380c2b50cce69" exitCode=0 Aug 04 12:33:50.027971 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:50.027615 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" event={"ID":"efcbc421-f0f2-4038-ad0d-2a212d591cd9","Type":"ContainerDied","Data":"4354f83cdc8998d4702aa1f12b3e3f3676db65466962fe03264380c2b50cce69"} Aug 04 12:33:51.187406 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.187378 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:51.304691 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.304661 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-bundle\") pod \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " Aug 04 12:33:51.304925 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.304723 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-util\") pod \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " Aug 04 12:33:51.304925 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.304745 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4z2d\" (UniqueName: \"kubernetes.io/projected/efcbc421-f0f2-4038-ad0d-2a212d591cd9-kube-api-access-z4z2d\") pod \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\" (UID: \"efcbc421-f0f2-4038-ad0d-2a212d591cd9\") " Aug 04 12:33:51.305452 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.305428 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-bundle" (OuterVolumeSpecName: "bundle") pod "efcbc421-f0f2-4038-ad0d-2a212d591cd9" (UID: "efcbc421-f0f2-4038-ad0d-2a212d591cd9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:33:51.307096 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.307068 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efcbc421-f0f2-4038-ad0d-2a212d591cd9-kube-api-access-z4z2d" (OuterVolumeSpecName: "kube-api-access-z4z2d") pod "efcbc421-f0f2-4038-ad0d-2a212d591cd9" (UID: "efcbc421-f0f2-4038-ad0d-2a212d591cd9"). InnerVolumeSpecName "kube-api-access-z4z2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:33:51.310620 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.310597 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-util" (OuterVolumeSpecName: "util") pod "efcbc421-f0f2-4038-ad0d-2a212d591cd9" (UID: "efcbc421-f0f2-4038-ad0d-2a212d591cd9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:33:51.406329 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.406289 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-util\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:51.406329 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.406323 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4z2d\" (UniqueName: \"kubernetes.io/projected/efcbc421-f0f2-4038-ad0d-2a212d591cd9-kube-api-access-z4z2d\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:51.406329 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:51.406336 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efcbc421-f0f2-4038-ad0d-2a212d591cd9-bundle\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:33:52.038075 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:52.038048 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" Aug 04 12:33:52.038238 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:52.038078 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7wmkj" event={"ID":"efcbc421-f0f2-4038-ad0d-2a212d591cd9","Type":"ContainerDied","Data":"5f2961615e90127f6ec12563d23154f275abcea11521ece4f192bf60b690a9d6"} Aug 04 12:33:52.038238 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:52.038124 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f2961615e90127f6ec12563d23154f275abcea11521ece4f192bf60b690a9d6" Aug 04 12:33:57.173875 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.173817 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g"] Aug 04 12:33:57.174480 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.174304 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerName="extract" Aug 04 12:33:57.174480 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.174323 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerName="extract" Aug 04 12:33:57.174480 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.174345 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerName="util" Aug 04 12:33:57.174480 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.174353 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerName="util" Aug 04 12:33:57.174480 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.174362 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerName="pull" Aug 04 12:33:57.174480 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.174370 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerName="pull" Aug 04 12:33:57.174480 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.174444 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="efcbc421-f0f2-4038-ad0d-2a212d591cd9" containerName="extract" Aug 04 12:33:57.178332 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.178311 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.182468 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.182451 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2pxdk\"" Aug 04 12:33:57.182913 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.182896 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 04 12:33:57.182913 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.182905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 04 12:33:57.183342 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.183328 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 04 12:33:57.190971 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.190948 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g"] Aug 04 12:33:57.249894 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.249862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4463d891-66a6-4633-a993-f384308925ec-metrics-cert\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.250030 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.249908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4463d891-66a6-4633-a993-f384308925ec-cert\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.250030 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.249929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sf8m\" (UniqueName: \"kubernetes.io/projected/4463d891-66a6-4633-a993-f384308925ec-kube-api-access-2sf8m\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.250030 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.249979 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4463d891-66a6-4633-a993-f384308925ec-manager-config\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.350927 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.350896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4463d891-66a6-4633-a993-f384308925ec-metrics-cert\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.351108 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.350945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4463d891-66a6-4633-a993-f384308925ec-cert\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.351108 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.350970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sf8m\" (UniqueName: \"kubernetes.io/projected/4463d891-66a6-4633-a993-f384308925ec-kube-api-access-2sf8m\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.351108 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.351030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4463d891-66a6-4633-a993-f384308925ec-manager-config\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.351780 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.351681 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4463d891-66a6-4633-a993-f384308925ec-manager-config\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.353556 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.353536 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4463d891-66a6-4633-a993-f384308925ec-cert\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.353662 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.353599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4463d891-66a6-4633-a993-f384308925ec-metrics-cert\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.361693 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.361671 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sf8m\" (UniqueName: \"kubernetes.io/projected/4463d891-66a6-4633-a993-f384308925ec-kube-api-access-2sf8m\") pod \"lws-controller-manager-c4cd7f74c-9xl8g\" (UID: \"4463d891-66a6-4633-a993-f384308925ec\") " pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.487878 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.487766 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:33:57.621865 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:57.621746 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g"] Aug 04 12:33:57.623958 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:33:57.623811 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4463d891_66a6_4633_a993_f384308925ec.slice/crio-64860ae4d314be6f005ad9bec9cac8f1436c1b5ed2ea358e07fa55c7ced92981 WatchSource:0}: Error finding container 64860ae4d314be6f005ad9bec9cac8f1436c1b5ed2ea358e07fa55c7ced92981: Status 404 returned error can't find the container with id 64860ae4d314be6f005ad9bec9cac8f1436c1b5ed2ea358e07fa55c7ced92981 Aug 04 12:33:58.068959 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:33:58.068927 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" event={"ID":"4463d891-66a6-4633-a993-f384308925ec","Type":"ContainerStarted","Data":"64860ae4d314be6f005ad9bec9cac8f1436c1b5ed2ea358e07fa55c7ced92981"} Aug 04 12:34:01.082958 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:01.082927 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" event={"ID":"4463d891-66a6-4633-a993-f384308925ec","Type":"ContainerStarted","Data":"8e5c966d5042ebf1ced11535156dae5009ee0369d812e14db9cf98431fc55594"} Aug 04 12:34:01.083353 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:01.083024 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:34:01.102687 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:01.102642 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" podStartSLOduration=0.786472124 podStartE2EDuration="4.10262574s" totalImagesPullingTime="3.316153616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:34:01.101009059 +0000 UTC m=+401.337440446" watchObservedRunningTime="2026-08-04 12:34:01.10262574 +0000 UTC m=+401.339057109" Aug 04 12:34:02.544345 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.544310 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk"] Aug 04 12:34:02.591681 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.591644 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk"] Aug 04 12:34:02.591853 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.591746 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.595584 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.595563 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 12:34:02.595718 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.595656 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 12:34:02.596043 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.596020 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gzmtm\"" Aug 04 12:34:02.596162 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.596043 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 12:34:02.596162 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.596122 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 12:34:02.687913 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.687879 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.688104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.687932 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-manifests\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.688104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.687968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-webhook-cert\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.688104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.688009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-tmp\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.688104 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.688062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6q5c\" (UniqueName: \"kubernetes.io/projected/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-kube-api-access-w6q5c\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.788621 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.788533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-manifests\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.788621 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.788570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-webhook-cert\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.788621 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.788588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-tmp\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.788621 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.788624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6q5c\" (UniqueName: \"kubernetes.io/projected/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-kube-api-access-w6q5c\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.788938 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.788662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.789064 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.789043 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-manifests\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.791073 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.791048 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-tmp\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.791274 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.791255 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.791385 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.791366 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-webhook-cert\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.798483 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.798465 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6q5c\" (UniqueName: \"kubernetes.io/projected/cffd9de0-48e0-4554-a8fc-1784fdc27d7b-kube-api-access-w6q5c\") pod \"opendatahub-operator-controller-manager-547f69df5b-65lmk\" (UID: \"cffd9de0-48e0-4554-a8fc-1784fdc27d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:02.902570 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:02.902515 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:03.038725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:03.038649 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk"] Aug 04 12:34:03.041720 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:34:03.041695 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcffd9de0_48e0_4554_a8fc_1784fdc27d7b.slice/crio-78caf080a21d7e6a551a14cd5b84c5b40503ed30b513c75b229a91e4fb5faed2 WatchSource:0}: Error finding container 78caf080a21d7e6a551a14cd5b84c5b40503ed30b513c75b229a91e4fb5faed2: Status 404 returned error can't find the container with id 78caf080a21d7e6a551a14cd5b84c5b40503ed30b513c75b229a91e4fb5faed2 Aug 04 12:34:03.092256 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:03.092227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" event={"ID":"cffd9de0-48e0-4554-a8fc-1784fdc27d7b","Type":"ContainerStarted","Data":"78caf080a21d7e6a551a14cd5b84c5b40503ed30b513c75b229a91e4fb5faed2"} Aug 04 12:34:06.111493 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:06.111456 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" event={"ID":"cffd9de0-48e0-4554-a8fc-1784fdc27d7b","Type":"ContainerStarted","Data":"d42040ff94e5d238a0fb61a3da119ccbd40567fc517a741a476f532851893a85"} Aug 04 12:34:07.117413 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:07.117372 2580 generic.go:356] "Generic (PLEG): container finished" podID="cffd9de0-48e0-4554-a8fc-1784fdc27d7b" containerID="d42040ff94e5d238a0fb61a3da119ccbd40567fc517a741a476f532851893a85" exitCode=0 Aug 04 12:34:07.117797 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:07.117458 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" event={"ID":"cffd9de0-48e0-4554-a8fc-1784fdc27d7b","Type":"ContainerDied","Data":"d42040ff94e5d238a0fb61a3da119ccbd40567fc517a741a476f532851893a85"} Aug 04 12:34:08.124639 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:08.124610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" event={"ID":"cffd9de0-48e0-4554-a8fc-1784fdc27d7b","Type":"ContainerStarted","Data":"27c2d4780933fbe72c294151266eca981909c2e129d8285ed4fe9c9b64e5a8a9"} Aug 04 12:34:08.125019 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:08.124737 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:08.148438 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:08.148391 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" podStartSLOduration=1.850134519 podStartE2EDuration="6.148377679s" totalImagesPullingTime="3.272692147s" totalInitContainerRuntime="1.025551013s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:34:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:34:08.146532215 +0000 UTC m=+408.382963594" watchObservedRunningTime="2026-08-04 12:34:08.148377679 +0000 UTC m=+408.384809107" Aug 04 12:34:12.089506 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:12.089476 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c4cd7f74c-9xl8g" Aug 04 12:34:19.132178 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:19.132144 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-547f69df5b-65lmk" Aug 04 12:34:29.874576 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:29.874543 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5"] Aug 04 12:34:29.879726 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:29.879710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:29.882848 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:29.882807 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Aug 04 12:34:29.882968 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:29.882847 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pxbkb\"" Aug 04 12:34:29.882968 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:29.882806 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Aug 04 12:34:29.882968 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:29.882817 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Aug 04 12:34:29.882968 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:29.882808 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 12:34:29.894603 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:29.894582 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5"] Aug 04 12:34:30.014979 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.014945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.015141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.014990 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5czs\" (UniqueName: \"kubernetes.io/projected/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-kube-api-access-w5czs\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.015141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.015053 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.015141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.015091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.015141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.015119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.015141 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.015139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.015321 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.015183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.116603 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.116572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.116792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.116626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.116792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.116663 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5czs\" (UniqueName: \"kubernetes.io/projected/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-kube-api-access-w5czs\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.116792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.116696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.116792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.116747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.116792 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.116787 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.117113 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.116815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.117328 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.117302 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.119265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.119238 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.119354 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.119277 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.119354 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.119347 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.119422 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.119350 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.126082 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.126032 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5czs\" (UniqueName: \"kubernetes.io/projected/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-kube-api-access-w5czs\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.126247 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.126232 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f58dcf7-d354-4ae7-ac32-6f4c515e77d5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pglv5\" (UID: \"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.188511 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.188473 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:30.325203 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:30.325177 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5"] Aug 04 12:34:30.328339 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:34:30.328306 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f58dcf7_d354_4ae7_ac32_6f4c515e77d5.slice/crio-e0f1f12086d8b73b7a45645c53779fc60d1a8f874ccaca5bf0a0a8c20f4cd3c7 WatchSource:0}: Error finding container e0f1f12086d8b73b7a45645c53779fc60d1a8f874ccaca5bf0a0a8c20f4cd3c7: Status 404 returned error can't find the container with id e0f1f12086d8b73b7a45645c53779fc60d1a8f874ccaca5bf0a0a8c20f4cd3c7 Aug 04 12:34:31.234608 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:31.234572 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" event={"ID":"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5","Type":"ContainerStarted","Data":"e0f1f12086d8b73b7a45645c53779fc60d1a8f874ccaca5bf0a0a8c20f4cd3c7"} Aug 04 12:34:32.704278 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:32.704224 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Aug 04 12:34:32.704601 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:32.704325 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Aug 04 12:34:33.246470 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:33.246431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" event={"ID":"9f58dcf7-d354-4ae7-ac32-6f4c515e77d5","Type":"ContainerStarted","Data":"1c71d2fffff3ff17dee3364109a0868c9215f026ea373ce90b5a657a51ebde5a"} Aug 04 12:34:33.246705 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:33.246663 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:34:33.249283 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:33.248952 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-pglv5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 12:34:33.249283 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:33.249034 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" podUID="9f58dcf7-d354-4ae7-ac32-6f4c515e77d5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 12:34:33.274495 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:33.274437 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" podStartSLOduration=1.901148055 podStartE2EDuration="4.274421425s" totalImagesPullingTime="2.37327337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:34:33.272673464 +0000 UTC m=+433.509104848" watchObservedRunningTime="2026-08-04 12:34:33.274421425 +0000 UTC m=+433.510852795" Aug 04 12:34:34.252018 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:34:34.251993 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pglv5" Aug 04 12:35:26.847732 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.847694 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7"] Aug 04 12:35:26.851448 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.851432 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.854109 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.854086 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 12:35:26.855157 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.855139 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2n8x9\"" Aug 04 12:35:26.855235 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.855174 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 12:35:26.861727 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.861704 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7"] Aug 04 12:35:26.873389 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.873366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.873526 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.873468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.873526 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.873494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqsdq\" (UniqueName: \"kubernetes.io/projected/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-kube-api-access-hqsdq\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.974253 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.974218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.974440 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.974274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.974440 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.974293 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqsdq\" (UniqueName: \"kubernetes.io/projected/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-kube-api-access-hqsdq\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.974673 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.974650 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.974745 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.974658 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.989801 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.989769 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5"] Aug 04 12:35:26.992202 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.992182 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqsdq\" (UniqueName: \"kubernetes.io/projected/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-kube-api-access-hqsdq\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:26.993412 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:26.993397 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.011301 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.011279 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5"] Aug 04 12:35:27.065914 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.065881 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj"] Aug 04 12:35:27.069266 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.069250 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.075090 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.075061 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.075207 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.075098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcl57\" (UniqueName: \"kubernetes.io/projected/2819dc9d-c30b-4a81-944c-c043b52b5cf3-kube-api-access-hcl57\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.075207 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.075167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.080633 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.080610 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj"] Aug 04 12:35:27.144678 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.144607 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp"] Aug 04 12:35:27.148080 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.148063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.157635 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.157599 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp"] Aug 04 12:35:27.160732 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.160708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:27.175897 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.175870 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.176028 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.175916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.176028 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.175969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcl57\" (UniqueName: \"kubernetes.io/projected/2819dc9d-c30b-4a81-944c-c043b52b5cf3-kube-api-access-hcl57\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.176028 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.176017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.176185 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.176045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.176185 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.176073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.176185 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.176134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7lb9\" (UniqueName: \"kubernetes.io/projected/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-kube-api-access-m7lb9\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.176185 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.176167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2b6h\" (UniqueName: \"kubernetes.io/projected/d3a84d52-61c7-427b-a830-a61f91db2e45-kube-api-access-n2b6h\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.176378 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.176211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.176458 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.176416 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.176555 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.176425 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.186012 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.185986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcl57\" (UniqueName: \"kubernetes.io/projected/2819dc9d-c30b-4a81-944c-c043b52b5cf3-kube-api-access-hcl57\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.277208 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.277383 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.277383 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277276 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.277383 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7lb9\" (UniqueName: \"kubernetes.io/projected/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-kube-api-access-m7lb9\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.277383 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2b6h\" (UniqueName: \"kubernetes.io/projected/d3a84d52-61c7-427b-a830-a61f91db2e45-kube-api-access-n2b6h\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.277631 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.277631 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277580 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.278137 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277766 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.278137 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.277818 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.278137 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.278074 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.288444 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.288421 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7lb9\" (UniqueName: \"kubernetes.io/projected/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-kube-api-access-m7lb9\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.288546 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.288488 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2b6h\" (UniqueName: \"kubernetes.io/projected/d3a84d52-61c7-427b-a830-a61f91db2e45-kube-api-access-n2b6h\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.299667 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.299642 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7"] Aug 04 12:35:27.301903 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:35:27.301879 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7bbc7d_1969_4d3c_8d4c_45c042bcef7d.slice/crio-f48cc74bc1510f3cae1d1d09826e54cc235d092cc6460681aeefafeec53dca86 WatchSource:0}: Error finding container f48cc74bc1510f3cae1d1d09826e54cc235d092cc6460681aeefafeec53dca86: Status 404 returned error can't find the container with id f48cc74bc1510f3cae1d1d09826e54cc235d092cc6460681aeefafeec53dca86 Aug 04 12:35:27.307042 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.307007 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:27.378689 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.378642 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:27.457862 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.457745 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:27.461275 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.461246 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5"] Aug 04 12:35:27.464807 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:35:27.464777 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2819dc9d_c30b_4a81_944c_c043b52b5cf3.slice/crio-c00df35c2d857a3774170e04ed4b7fdd8b4abd0b30d80c0280d20274bdd58416 WatchSource:0}: Error finding container c00df35c2d857a3774170e04ed4b7fdd8b4abd0b30d80c0280d20274bdd58416: Status 404 returned error can't find the container with id c00df35c2d857a3774170e04ed4b7fdd8b4abd0b30d80c0280d20274bdd58416 Aug 04 12:35:27.491322 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.491293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" event={"ID":"2819dc9d-c30b-4a81-944c-c043b52b5cf3","Type":"ContainerStarted","Data":"c00df35c2d857a3774170e04ed4b7fdd8b4abd0b30d80c0280d20274bdd58416"} Aug 04 12:35:27.493243 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.493164 2580 generic.go:356] "Generic (PLEG): container finished" podID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerID="a0c3455ecbf0469e6e75ebd13f88547c85e022b0fcd8cf825c738891036d793f" exitCode=0 Aug 04 12:35:27.493307 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.493268 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" event={"ID":"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d","Type":"ContainerDied","Data":"a0c3455ecbf0469e6e75ebd13f88547c85e022b0fcd8cf825c738891036d793f"} Aug 04 12:35:27.493355 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.493309 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" event={"ID":"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d","Type":"ContainerStarted","Data":"f48cc74bc1510f3cae1d1d09826e54cc235d092cc6460681aeefafeec53dca86"} Aug 04 12:35:27.545957 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.545733 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj"] Aug 04 12:35:27.548497 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:35:27.548463 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a84d52_61c7_427b_a830_a61f91db2e45.slice/crio-dc0aca1e26448ea38a53e33803bae270af3cd98347c2a34eb618202783494e08 WatchSource:0}: Error finding container dc0aca1e26448ea38a53e33803bae270af3cd98347c2a34eb618202783494e08: Status 404 returned error can't find the container with id dc0aca1e26448ea38a53e33803bae270af3cd98347c2a34eb618202783494e08 Aug 04 12:35:27.624595 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:27.624545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp"] Aug 04 12:35:27.636611 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:35:27.636586 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce16ab93_4cb7_4e03_958f_dc78c44ceb9b.slice/crio-0a2581dd1e1634028a61880b770e8c069f738ec69ae118a932342a664637653f WatchSource:0}: Error finding container 0a2581dd1e1634028a61880b770e8c069f738ec69ae118a932342a664637653f: Status 404 returned error can't find the container with id 0a2581dd1e1634028a61880b770e8c069f738ec69ae118a932342a664637653f Aug 04 12:35:28.499159 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.499125 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerID="6e1f7ef1199191ceaf98cdc03e8b24ebb0fe085bb4318faaf78be830dbb177ce" exitCode=0 Aug 04 12:35:28.499755 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.499208 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" event={"ID":"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b","Type":"ContainerDied","Data":"6e1f7ef1199191ceaf98cdc03e8b24ebb0fe085bb4318faaf78be830dbb177ce"} Aug 04 12:35:28.499755 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.499262 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" event={"ID":"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b","Type":"ContainerStarted","Data":"0a2581dd1e1634028a61880b770e8c069f738ec69ae118a932342a664637653f"} Aug 04 12:35:28.500753 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.500731 2580 generic.go:356] "Generic (PLEG): container finished" podID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerID="2beffc58fb657568e81a96cb86634749b6d67f46290abb18c3b73f4c4d33b144" exitCode=0 Aug 04 12:35:28.500822 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.500763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" event={"ID":"2819dc9d-c30b-4a81-944c-c043b52b5cf3","Type":"ContainerDied","Data":"2beffc58fb657568e81a96cb86634749b6d67f46290abb18c3b73f4c4d33b144"} Aug 04 12:35:28.502822 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.502793 2580 generic.go:356] "Generic (PLEG): container finished" podID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerID="f477430c50dd894f6451f9e30c3f0d960b81d210ed5cdd23696d65d1e9fd2e3b" exitCode=0 Aug 04 12:35:28.502925 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.502821 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" event={"ID":"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d","Type":"ContainerDied","Data":"f477430c50dd894f6451f9e30c3f0d960b81d210ed5cdd23696d65d1e9fd2e3b"} Aug 04 12:35:28.504899 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.504879 2580 generic.go:356] "Generic (PLEG): container finished" podID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerID="722fbb9608fc0f1ee66a9c3e3a56e0990876ee7b96bf7aba078b20739496ee09" exitCode=0 Aug 04 12:35:28.504993 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.504963 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" event={"ID":"d3a84d52-61c7-427b-a830-a61f91db2e45","Type":"ContainerDied","Data":"722fbb9608fc0f1ee66a9c3e3a56e0990876ee7b96bf7aba078b20739496ee09"} Aug 04 12:35:28.505047 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:28.504997 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" event={"ID":"d3a84d52-61c7-427b-a830-a61f91db2e45","Type":"ContainerStarted","Data":"dc0aca1e26448ea38a53e33803bae270af3cd98347c2a34eb618202783494e08"} Aug 04 12:35:29.511645 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:29.511569 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerID="7a27509a090192d7747d078993f399ea420d2c4f5be0a2709f232e50e5f3492f" exitCode=0 Aug 04 12:35:29.512063 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:29.511655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" event={"ID":"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b","Type":"ContainerDied","Data":"7a27509a090192d7747d078993f399ea420d2c4f5be0a2709f232e50e5f3492f"} Aug 04 12:35:29.513416 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:29.513396 2580 generic.go:356] "Generic (PLEG): container finished" podID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerID="5c57d0ea0bd52796cdabe0aad99010b82bb3c9fe41a972708eb84a2156d5271d" exitCode=0 Aug 04 12:35:29.513493 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:29.513478 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" event={"ID":"2819dc9d-c30b-4a81-944c-c043b52b5cf3","Type":"ContainerDied","Data":"5c57d0ea0bd52796cdabe0aad99010b82bb3c9fe41a972708eb84a2156d5271d"} Aug 04 12:35:29.515413 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:29.515393 2580 generic.go:356] "Generic (PLEG): container finished" podID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerID="40dccf4a6e865a5d7c0c3bb1a69748968738425b3cd8999fb2cab32e4c5b7857" exitCode=0 Aug 04 12:35:29.515501 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:29.515441 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" event={"ID":"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d","Type":"ContainerDied","Data":"40dccf4a6e865a5d7c0c3bb1a69748968738425b3cd8999fb2cab32e4c5b7857"} Aug 04 12:35:30.522363 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.522327 2580 generic.go:356] "Generic (PLEG): container finished" podID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerID="db428d6d0679e4799b10a91d6b5c10a9fd82311c783892850e2a39197dc46eed" exitCode=0 Aug 04 12:35:30.522737 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.522415 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" event={"ID":"2819dc9d-c30b-4a81-944c-c043b52b5cf3","Type":"ContainerDied","Data":"db428d6d0679e4799b10a91d6b5c10a9fd82311c783892850e2a39197dc46eed"} Aug 04 12:35:30.524016 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.523996 2580 generic.go:356] "Generic (PLEG): container finished" podID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerID="31e36fcdffc94681c137a382b9a8a1011c5526bd95196fb6fa19189180f0b0ca" exitCode=0 Aug 04 12:35:30.524132 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.524076 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" event={"ID":"d3a84d52-61c7-427b-a830-a61f91db2e45","Type":"ContainerDied","Data":"31e36fcdffc94681c137a382b9a8a1011c5526bd95196fb6fa19189180f0b0ca"} Aug 04 12:35:30.526434 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.526413 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerID="7d98a41b1467636e2745961d618a2a54d0699b8b1839433c95a6d2d0d74df8ba" exitCode=0 Aug 04 12:35:30.526533 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.526482 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" event={"ID":"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b","Type":"ContainerDied","Data":"7d98a41b1467636e2745961d618a2a54d0699b8b1839433c95a6d2d0d74df8ba"} Aug 04 12:35:30.666008 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.665977 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:30.709034 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.709003 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-util\") pod \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " Aug 04 12:35:30.709189 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.709173 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-bundle\") pod \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " Aug 04 12:35:30.709232 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.709218 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqsdq\" (UniqueName: \"kubernetes.io/projected/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-kube-api-access-hqsdq\") pod \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\" (UID: \"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d\") " Aug 04 12:35:30.709728 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.709688 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-bundle" (OuterVolumeSpecName: "bundle") pod "4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" (UID: "4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:35:30.711367 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.711339 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-kube-api-access-hqsdq" (OuterVolumeSpecName: "kube-api-access-hqsdq") pod "4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" (UID: "4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d"). InnerVolumeSpecName "kube-api-access-hqsdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:35:30.714009 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.713987 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-util" (OuterVolumeSpecName: "util") pod "4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" (UID: "4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:35:30.810850 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.810796 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-bundle\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:30.810850 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.810849 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqsdq\" (UniqueName: \"kubernetes.io/projected/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-kube-api-access-hqsdq\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:30.811031 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:30.810861 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d-util\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:31.533386 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.533353 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" event={"ID":"4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d","Type":"ContainerDied","Data":"f48cc74bc1510f3cae1d1d09826e54cc235d092cc6460681aeefafeec53dca86"} Aug 04 12:35:31.533778 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.533427 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f48cc74bc1510f3cae1d1d09826e54cc235d092cc6460681aeefafeec53dca86" Aug 04 12:35:31.533778 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.533360 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a05gqn7" Aug 04 12:35:31.535435 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.535408 2580 generic.go:356] "Generic (PLEG): container finished" podID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerID="1a56f2799dbc9b056b78e7c796216976bf6afb2d6a9edf54fac4b9624c02d332" exitCode=0 Aug 04 12:35:31.535560 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.535479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" event={"ID":"d3a84d52-61c7-427b-a830-a61f91db2e45","Type":"ContainerDied","Data":"1a56f2799dbc9b056b78e7c796216976bf6afb2d6a9edf54fac4b9624c02d332"} Aug 04 12:35:31.688032 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.688008 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:31.717452 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.717429 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:31.717607 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.717508 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7lb9\" (UniqueName: \"kubernetes.io/projected/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-kube-api-access-m7lb9\") pod \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " Aug 04 12:35:31.717667 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.717604 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-bundle\") pod \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " Aug 04 12:35:31.717725 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.717677 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-util\") pod \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\" (UID: \"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b\") " Aug 04 12:35:31.718147 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.718119 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-bundle" (OuterVolumeSpecName: "bundle") pod "ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" (UID: "ce16ab93-4cb7-4e03-958f-dc78c44ceb9b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:35:31.719758 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.719734 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-kube-api-access-m7lb9" (OuterVolumeSpecName: "kube-api-access-m7lb9") pod "ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" (UID: "ce16ab93-4cb7-4e03-958f-dc78c44ceb9b"). InnerVolumeSpecName "kube-api-access-m7lb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:35:31.722841 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.722811 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-util" (OuterVolumeSpecName: "util") pod "ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" (UID: "ce16ab93-4cb7-4e03-958f-dc78c44ceb9b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:35:31.819112 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.819084 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-util\") pod \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " Aug 04 12:35:31.819270 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.819148 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-bundle\") pod \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " Aug 04 12:35:31.819270 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.819209 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcl57\" (UniqueName: \"kubernetes.io/projected/2819dc9d-c30b-4a81-944c-c043b52b5cf3-kube-api-access-hcl57\") pod \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\" (UID: \"2819dc9d-c30b-4a81-944c-c043b52b5cf3\") " Aug 04 12:35:31.819426 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.819410 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-util\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:31.819479 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.819433 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7lb9\" (UniqueName: \"kubernetes.io/projected/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-kube-api-access-m7lb9\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:31.819479 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.819448 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce16ab93-4cb7-4e03-958f-dc78c44ceb9b-bundle\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:31.819650 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.819634 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-bundle" (OuterVolumeSpecName: "bundle") pod "2819dc9d-c30b-4a81-944c-c043b52b5cf3" (UID: "2819dc9d-c30b-4a81-944c-c043b52b5cf3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:35:31.821629 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.821602 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2819dc9d-c30b-4a81-944c-c043b52b5cf3-kube-api-access-hcl57" (OuterVolumeSpecName: "kube-api-access-hcl57") pod "2819dc9d-c30b-4a81-944c-c043b52b5cf3" (UID: "2819dc9d-c30b-4a81-944c-c043b52b5cf3"). InnerVolumeSpecName "kube-api-access-hcl57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:35:31.824566 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.824545 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-util" (OuterVolumeSpecName: "util") pod "2819dc9d-c30b-4a81-944c-c043b52b5cf3" (UID: "2819dc9d-c30b-4a81-944c-c043b52b5cf3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:35:31.920189 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.920157 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-util\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:31.920189 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.920184 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819dc9d-c30b-4a81-944c-c043b52b5cf3-bundle\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:31.920189 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:31.920195 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcl57\" (UniqueName: \"kubernetes.io/projected/2819dc9d-c30b-4a81-944c-c043b52b5cf3-kube-api-access-hcl57\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:32.542012 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.541927 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" Aug 04 12:35:32.542012 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.541946 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbqqxnp" event={"ID":"ce16ab93-4cb7-4e03-958f-dc78c44ceb9b","Type":"ContainerDied","Data":"0a2581dd1e1634028a61880b770e8c069f738ec69ae118a932342a664637653f"} Aug 04 12:35:32.542012 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.541988 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a2581dd1e1634028a61880b770e8c069f738ec69ae118a932342a664637653f" Aug 04 12:35:32.543891 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.543864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" event={"ID":"2819dc9d-c30b-4a81-944c-c043b52b5cf3","Type":"ContainerDied","Data":"c00df35c2d857a3774170e04ed4b7fdd8b4abd0b30d80c0280d20274bdd58416"} Aug 04 12:35:32.543891 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.543893 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c8gmj5" Aug 04 12:35:32.544084 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.543905 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c00df35c2d857a3774170e04ed4b7fdd8b4abd0b30d80c0280d20274bdd58416" Aug 04 12:35:32.678404 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.678381 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:32.726939 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.726909 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-util\") pod \"d3a84d52-61c7-427b-a830-a61f91db2e45\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " Aug 04 12:35:32.727112 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.726960 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-bundle\") pod \"d3a84d52-61c7-427b-a830-a61f91db2e45\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " Aug 04 12:35:32.727112 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.726998 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2b6h\" (UniqueName: \"kubernetes.io/projected/d3a84d52-61c7-427b-a830-a61f91db2e45-kube-api-access-n2b6h\") pod \"d3a84d52-61c7-427b-a830-a61f91db2e45\" (UID: \"d3a84d52-61c7-427b-a830-a61f91db2e45\") " Aug 04 12:35:32.727613 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.727583 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-bundle" (OuterVolumeSpecName: "bundle") pod "d3a84d52-61c7-427b-a830-a61f91db2e45" (UID: "d3a84d52-61c7-427b-a830-a61f91db2e45"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:35:32.729778 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.729748 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3a84d52-61c7-427b-a830-a61f91db2e45-kube-api-access-n2b6h" (OuterVolumeSpecName: "kube-api-access-n2b6h") pod "d3a84d52-61c7-427b-a830-a61f91db2e45" (UID: "d3a84d52-61c7-427b-a830-a61f91db2e45"). InnerVolumeSpecName "kube-api-access-n2b6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:35:32.732536 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.732511 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-util" (OuterVolumeSpecName: "util") pod "d3a84d52-61c7-427b-a830-a61f91db2e45" (UID: "d3a84d52-61c7-427b-a830-a61f91db2e45"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:35:32.827587 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.827559 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-util\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:32.827587 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.827585 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3a84d52-61c7-427b-a830-a61f91db2e45-bundle\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:32.827775 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:32.827593 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2b6h\" (UniqueName: \"kubernetes.io/projected/d3a84d52-61c7-427b-a830-a61f91db2e45-kube-api-access-n2b6h\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:35:33.550451 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:33.550422 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" Aug 04 12:35:33.550962 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:33.550448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870532h6wj" event={"ID":"d3a84d52-61c7-427b-a830-a61f91db2e45","Type":"ContainerDied","Data":"dc0aca1e26448ea38a53e33803bae270af3cd98347c2a34eb618202783494e08"} Aug 04 12:35:33.550962 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:35:33.550500 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc0aca1e26448ea38a53e33803bae270af3cd98347c2a34eb618202783494e08" Aug 04 12:36:52.039873 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.039820 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040259 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerName="extract" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040276 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerName="extract" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040291 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerName="pull" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040301 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerName="pull" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040312 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerName="util" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040321 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerName="util" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040332 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerName="pull" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040343 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerName="pull" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040353 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerName="util" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040361 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerName="util" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040370 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerName="pull" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040377 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerName="pull" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040398 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerName="util" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040407 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerName="util" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040417 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerName="extract" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040426 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerName="extract" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040440 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerName="util" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040452 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerName="util" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040463 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerName="extract" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040470 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerName="extract" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040483 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerName="pull" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040490 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerName="pull" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040504 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerName="extract" Aug 04 12:36:52.040527 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040512 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerName="extract" Aug 04 12:36:52.041373 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040597 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b7bbc7d-1969-4d3c-8d4c-45c042bcef7d" containerName="extract" Aug 04 12:36:52.041373 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040606 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3a84d52-61c7-427b-a830-a61f91db2e45" containerName="extract" Aug 04 12:36:52.041373 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040612 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce16ab93-4cb7-4e03-958f-dc78c44ceb9b" containerName="extract" Aug 04 12:36:52.041373 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.040618 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2819dc9d-c30b-4a81-944c-c043b52b5cf3" containerName="extract" Aug 04 12:36:52.043503 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.043487 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:52.046086 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.046068 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 12:36:52.046552 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.046535 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 12:36:52.046615 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.046598 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Aug 04 12:36:52.047211 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.047192 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rvx5k\"" Aug 04 12:36:52.054088 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.054065 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:36:52.082295 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.082263 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:36:52.179667 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.179626 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfpmn\" (UniqueName: \"kubernetes.io/projected/a99054ca-68ee-4530-b475-a7c17302681c-kube-api-access-jfpmn\") pod \"limitador-limitador-85d9db4d5d-8qmrl\" (UID: \"a99054ca-68ee-4530-b475-a7c17302681c\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:52.179667 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.179672 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a99054ca-68ee-4530-b475-a7c17302681c-config-file\") pod \"limitador-limitador-85d9db4d5d-8qmrl\" (UID: \"a99054ca-68ee-4530-b475-a7c17302681c\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:52.280933 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.280901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a99054ca-68ee-4530-b475-a7c17302681c-config-file\") pod \"limitador-limitador-85d9db4d5d-8qmrl\" (UID: \"a99054ca-68ee-4530-b475-a7c17302681c\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:52.281136 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.281005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfpmn\" (UniqueName: \"kubernetes.io/projected/a99054ca-68ee-4530-b475-a7c17302681c-kube-api-access-jfpmn\") pod \"limitador-limitador-85d9db4d5d-8qmrl\" (UID: \"a99054ca-68ee-4530-b475-a7c17302681c\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:52.281526 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.281505 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a99054ca-68ee-4530-b475-a7c17302681c-config-file\") pod \"limitador-limitador-85d9db4d5d-8qmrl\" (UID: \"a99054ca-68ee-4530-b475-a7c17302681c\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:52.290298 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.290238 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfpmn\" (UniqueName: \"kubernetes.io/projected/a99054ca-68ee-4530-b475-a7c17302681c-kube-api-access-jfpmn\") pod \"limitador-limitador-85d9db4d5d-8qmrl\" (UID: \"a99054ca-68ee-4530-b475-a7c17302681c\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:52.354581 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.354532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:52.493156 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.493135 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:36:52.494933 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:36:52.494905 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99054ca_68ee_4530_b475_a7c17302681c.slice/crio-a7c4737dc1d1f3d624474ec145c10a0e7aaff82beba301942c44c739d3a84a11 WatchSource:0}: Error finding container a7c4737dc1d1f3d624474ec145c10a0e7aaff82beba301942c44c739d3a84a11: Status 404 returned error can't find the container with id a7c4737dc1d1f3d624474ec145c10a0e7aaff82beba301942c44c739d3a84a11 Aug 04 12:36:52.947765 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:52.947728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" event={"ID":"a99054ca-68ee-4530-b475-a7c17302681c","Type":"ContainerStarted","Data":"a7c4737dc1d1f3d624474ec145c10a0e7aaff82beba301942c44c739d3a84a11"} Aug 04 12:36:55.964501 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:55.964468 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" event={"ID":"a99054ca-68ee-4530-b475-a7c17302681c","Type":"ContainerStarted","Data":"b2b18dd62ed9ef64245d65ae0c4ae557c6bc9f9425a46e336671f68fbe2ee452"} Aug 04 12:36:55.964931 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:55.964604 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:36:55.984770 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:36:55.984721 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" podStartSLOduration=1.500008335 podStartE2EDuration="3.984704375s" totalImagesPullingTime="2.48469604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:36:55.984084096 +0000 UTC m=+576.220515465" watchObservedRunningTime="2026-08-04 12:36:55.984704375 +0000 UTC m=+576.221135745" Aug 04 12:37:06.970882 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:37:06.970818 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl" Aug 04 12:37:20.214462 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:37:20.214436 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:37:20.216129 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:37:20.216109 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:38:10.237872 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:38:10.237818 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:38:25.499301 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:38:25.499263 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:38:32.498383 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:38:32.498350 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:39:06.601055 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:06.601018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:39:17.898593 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.898518 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps"] Aug 04 12:39:17.901954 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.901929 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:17.905596 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.905572 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 12:39:17.905726 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.905597 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 12:39:17.905726 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.905578 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mddbc\"" Aug 04 12:39:17.905726 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.905634 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Aug 04 12:39:17.912984 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.912959 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps"] Aug 04 12:39:17.927107 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.927067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80901ac2-7f0d-4b18-9ded-81f355bfcc78-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:17.927257 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.927133 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:17.927257 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.927189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:17.927257 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.927232 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvh5n\" (UniqueName: \"kubernetes.io/projected/80901ac2-7f0d-4b18-9ded-81f355bfcc78-kube-api-access-nvh5n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:17.927386 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.927307 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:17.927386 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:17.927337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028007 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.027967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80901ac2-7f0d-4b18-9ded-81f355bfcc78-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028201 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.028020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028201 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.028051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028201 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.028091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvh5n\" (UniqueName: \"kubernetes.io/projected/80901ac2-7f0d-4b18-9ded-81f355bfcc78-kube-api-access-nvh5n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028201 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.028148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028201 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.028175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028526 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.028505 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028581 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.028524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.028650 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.028620 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.030407 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.030384 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80901ac2-7f0d-4b18-9ded-81f355bfcc78-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.030601 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.030585 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80901ac2-7f0d-4b18-9ded-81f355bfcc78-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.038481 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.038452 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvh5n\" (UniqueName: \"kubernetes.io/projected/80901ac2-7f0d-4b18-9ded-81f355bfcc78-kube-api-access-nvh5n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps\" (UID: \"80901ac2-7f0d-4b18-9ded-81f355bfcc78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.212759 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.212673 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:18.351847 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.351797 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps"] Aug 04 12:39:18.353400 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:39:18.353377 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80901ac2_7f0d_4b18_9ded_81f355bfcc78.slice/crio-88ef473f44c6f660103eacd3b331d1b506f3e72d936a3952926c3b0c8eb38b7b WatchSource:0}: Error finding container 88ef473f44c6f660103eacd3b331d1b506f3e72d936a3952926c3b0c8eb38b7b: Status 404 returned error can't find the container with id 88ef473f44c6f660103eacd3b331d1b506f3e72d936a3952926c3b0c8eb38b7b Aug 04 12:39:18.354818 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.354803 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 12:39:18.666390 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:18.666343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" event={"ID":"80901ac2-7f0d-4b18-9ded-81f355bfcc78","Type":"ContainerStarted","Data":"88ef473f44c6f660103eacd3b331d1b506f3e72d936a3952926c3b0c8eb38b7b"} Aug 04 12:39:19.100555 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:19.100520 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:39:19.673256 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:19.673171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" event={"ID":"80901ac2-7f0d-4b18-9ded-81f355bfcc78","Type":"ContainerStarted","Data":"c4acf825b0e7b0eaf2113f0babb8b58f242b363f9c313e708b0a5b1958bcca7b"} Aug 04 12:39:19.694472 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:19.694412 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" podStartSLOduration=1.668151016 podStartE2EDuration="2.694390795s" totalImagesPullingTime="1.026239779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:39:19.693719093 +0000 UTC m=+719.930150464" watchObservedRunningTime="2026-08-04 12:39:19.694390795 +0000 UTC m=+719.930822166" Aug 04 12:39:23.804433 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:23.804401 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:39:28.213180 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:28.213140 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:28.213580 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:28.213221 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:28.225713 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:28.225683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:28.734589 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:28.734557 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps" Aug 04 12:39:40.102639 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:39:40.102598 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:41:31.873870 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:41:31.873817 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:41:40.388626 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:41:40.388598 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:41:51.903531 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:41:51.903497 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:42:20.255975 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:42:20.255897 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:42:20.256517 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:42:20.256125 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:43:31.604724 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:43:31.604685 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:43:39.795143 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:43:39.795109 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:44:36.893977 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:44:36.893938 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:44:43.999151 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:44:43.999113 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:44:59.501132 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:44:59.501086 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:45:05.395684 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:45:05.395649 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:45:14.899178 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:45:14.899097 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:45:21.594481 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:45:21.594439 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:45:35.499787 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:45:35.499754 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:45:40.598413 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:45:40.598379 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:45:45.694931 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:45:45.694893 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:45:51.698075 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:45:51.698021 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:45:56.695225 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:45:56.695189 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:46:07.697730 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:46:07.697683 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:46:18.398051 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:46:18.398022 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:46:41.602569 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:46:41.602528 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:46:51.998676 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:46:51.998644 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:46:58.603850 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:46:58.603800 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:47:06.905682 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:06.905643 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:47:17.596356 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:17.596312 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:47:20.290219 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:20.290189 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:47:20.290672 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:20.290364 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:47:23.393861 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:23.393811 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:47:31.998520 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:31.998480 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:47:40.401918 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:40.401884 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:47:52.195355 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:52.195317 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:47:58.896660 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:47:58.896621 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:48:03.993609 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:48:03.993574 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:48:10.101279 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:48:10.101241 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:48:21.090684 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:48:21.090596 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:48:26.000364 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:48:26.000328 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:48:37.108349 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:48:37.108317 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:50:57.606215 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:50:57.606137 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:51:04.706324 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:51:04.706292 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:51:26.811455 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:51:26.811365 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:51:58.540092 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:51:58.540057 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:52:04.719955 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:04.719918 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:52:13.598918 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:13.598878 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:52:20.109334 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:20.109296 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:52:20.322191 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:20.322161 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:52:20.325195 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:20.325171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:52:29.099475 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:29.099437 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:52:34.808265 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:34.808227 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:52:44.428464 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:44.428428 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:52:49.412652 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:49.412614 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:52:59.694100 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:52:59.694062 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:53:06.301878 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:53:06.301824 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:53:15.503892 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:53:15.503858 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:53:37.709162 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:53:37.709126 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:53:44.599468 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:53:44.599437 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:53:51.697872 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:53:51.697817 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:54:05.096548 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:54:05.096517 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:54:43.131421 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:54:43.131383 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:54:53.611094 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:54:53.611058 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:54:58.829066 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:54:58.829026 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:55:05.204585 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:55:05.204552 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:55:15.800053 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:55:15.800015 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:55:22.106402 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:55:22.106365 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:55:29.805812 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:55:29.805779 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:55:38.000156 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:55:38.000121 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:55:48.315708 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:55:48.315626 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:55:54.849538 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:55:54.849501 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:56:10.807270 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:56:10.807236 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:56:17.319931 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:56:17.319894 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:56:29.309499 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:56:29.309464 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:56:35.893920 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:56:35.893883 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:56:44.204297 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:56:44.204263 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:56:48.295378 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:56:48.295342 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:56:54.406304 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:56:54.406261 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:57:04.621574 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:57:04.621536 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:57:10.807904 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:57:10.807870 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:57:20.354302 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:57:20.354274 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:57:20.362293 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:57:20.362270 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 12:57:21.800322 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:57:21.800291 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:57:28.043306 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:57:28.043274 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:57:39.010000 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:57:39.009964 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:57:54.433098 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:57:54.433061 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:58:00.493150 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:58:00.493109 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:58:07.215540 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:58:07.215506 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:58:18.425616 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:58:18.425584 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:58:26.707381 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:58:26.707348 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:58:32.310234 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:58:32.310196 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:58:47.003907 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:58:47.003810 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:59:08.622918 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.622881 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg"] Aug 04 12:59:08.626390 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.626373 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" Aug 04 12:59:08.629196 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.629171 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 12:59:08.629317 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.629272 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 12:59:08.635460 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.635434 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg"] Aug 04 12:59:08.762801 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.762762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066b2c69-e3b0-41c8-a71c-1f19bfab8443-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1756385-t96kg\" (UID: \"066b2c69-e3b0-41c8-a71c-1f19bfab8443\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" Aug 04 12:59:08.863898 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.863857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066b2c69-e3b0-41c8-a71c-1f19bfab8443-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1756385-t96kg\" (UID: \"066b2c69-e3b0-41c8-a71c-1f19bfab8443\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" Aug 04 12:59:08.864542 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.864522 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066b2c69-e3b0-41c8-a71c-1f19bfab8443-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1756385-t96kg\" (UID: \"066b2c69-e3b0-41c8-a71c-1f19bfab8443\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" Aug 04 12:59:08.937889 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:08.937802 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" Aug 04 12:59:09.071554 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:09.071529 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg"] Aug 04 12:59:09.073328 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:59:09.073296 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066b2c69_e3b0_41c8_a71c_1f19bfab8443.slice/crio-e66e0529546c013abfe9ba64dfde9c7758f5c3fb34bea1bc1035e0934934e29d WatchSource:0}: Error finding container e66e0529546c013abfe9ba64dfde9c7758f5c3fb34bea1bc1035e0934934e29d: Status 404 returned error can't find the container with id e66e0529546c013abfe9ba64dfde9c7758f5c3fb34bea1bc1035e0934934e29d Aug 04 12:59:09.075307 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:09.075288 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 12:59:09.674000 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:09.673961 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" event={"ID":"066b2c69-e3b0-41c8-a71c-1f19bfab8443","Type":"ContainerStarted","Data":"e66e0529546c013abfe9ba64dfde9c7758f5c3fb34bea1bc1035e0934934e29d"} Aug 04 12:59:12.692903 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:12.692859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" event={"ID":"066b2c69-e3b0-41c8-a71c-1f19bfab8443","Type":"ContainerStarted","Data":"a5adb207f801303fe8a00389d10ec3a9f1b3cb2690aacaecfa4b613fb594f90c"} Aug 04 12:59:12.712572 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:12.712514 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" podStartSLOduration=1.630635237 podStartE2EDuration="4.712497421s" totalImagesPullingTime="3.081862184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:59:12.710198376 +0000 UTC m=+1912.946629749" watchObservedRunningTime="2026-08-04 12:59:12.712497421 +0000 UTC m=+1912.948928791" Aug 04 12:59:13.700072 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:13.700032 2580 generic.go:356] "Generic (PLEG): container finished" podID="066b2c69-e3b0-41c8-a71c-1f19bfab8443" containerID="a5adb207f801303fe8a00389d10ec3a9f1b3cb2690aacaecfa4b613fb594f90c" exitCode=0 Aug 04 12:59:13.700486 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:13.700116 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" event={"ID":"066b2c69-e3b0-41c8-a71c-1f19bfab8443","Type":"ContainerDied","Data":"a5adb207f801303fe8a00389d10ec3a9f1b3cb2690aacaecfa4b613fb594f90c"} Aug 04 12:59:14.839135 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:14.839105 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" Aug 04 12:59:14.914568 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:14.914533 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066b2c69-e3b0-41c8-a71c-1f19bfab8443-maas-api-service-ca\") pod \"066b2c69-e3b0-41c8-a71c-1f19bfab8443\" (UID: \"066b2c69-e3b0-41c8-a71c-1f19bfab8443\") " Aug 04 12:59:14.914793 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:14.914772 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/066b2c69-e3b0-41c8-a71c-1f19bfab8443-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "066b2c69-e3b0-41c8-a71c-1f19bfab8443" (UID: "066b2c69-e3b0-41c8-a71c-1f19bfab8443"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:59:15.015593 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:15.015515 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066b2c69-e3b0-41c8-a71c-1f19bfab8443-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 12:59:15.712016 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:15.711979 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" event={"ID":"066b2c69-e3b0-41c8-a71c-1f19bfab8443","Type":"ContainerDied","Data":"e66e0529546c013abfe9ba64dfde9c7758f5c3fb34bea1bc1035e0934934e29d"} Aug 04 12:59:15.712016 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:15.712015 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg" Aug 04 12:59:15.712232 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:15.712032 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e66e0529546c013abfe9ba64dfde9c7758f5c3fb34bea1bc1035e0934934e29d" Aug 04 12:59:25.030600 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:25.030526 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg"] Aug 04 12:59:25.032652 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:25.032627 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1756385-t96kg"] Aug 04 12:59:26.276800 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:26.276762 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="066b2c69-e3b0-41c8-a71c-1f19bfab8443" path="/var/lib/kubelet/pods/066b2c69-e3b0-41c8-a71c-1f19bfab8443/volumes" Aug 04 12:59:27.205698 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:27.205667 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:59:35.794669 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:35.794611 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 12:59:57.478557 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.478520 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8"] Aug 04 12:59:57.479060 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.478896 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="066b2c69-e3b0-41c8-a71c-1f19bfab8443" containerName="revoke-keys" Aug 04 12:59:57.479060 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.478910 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="066b2c69-e3b0-41c8-a71c-1f19bfab8443" containerName="revoke-keys" Aug 04 12:59:57.479060 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.479010 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="066b2c69-e3b0-41c8-a71c-1f19bfab8443" containerName="revoke-keys" Aug 04 12:59:57.482117 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.482097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" Aug 04 12:59:57.484559 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.484533 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 12:59:57.484671 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.484592 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 12:59:57.490773 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.490747 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8"] Aug 04 12:59:57.555282 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.555246 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6849c27f-1f9b-4dab-9a85-9b74013521d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8\" (UID: \"6849c27f-1f9b-4dab-9a85-9b74013521d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" Aug 04 12:59:57.657018 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.656973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6849c27f-1f9b-4dab-9a85-9b74013521d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8\" (UID: \"6849c27f-1f9b-4dab-9a85-9b74013521d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" Aug 04 12:59:57.657797 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.657736 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6849c27f-1f9b-4dab-9a85-9b74013521d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8\" (UID: \"6849c27f-1f9b-4dab-9a85-9b74013521d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" Aug 04 12:59:57.794436 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.794349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" Aug 04 12:59:57.928767 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:57.928737 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8"] Aug 04 12:59:57.930626 ip-10-0-132-96 kubenswrapper[2580]: W0804 12:59:57.930603 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6849c27f_1f9b_4dab_9a85_9b74013521d9.slice/crio-0179a26c8877d9ed1297c352f5e01488dfa7f91561ee6db89fff407986063866 WatchSource:0}: Error finding container 0179a26c8877d9ed1297c352f5e01488dfa7f91561ee6db89fff407986063866: Status 404 returned error can't find the container with id 0179a26c8877d9ed1297c352f5e01488dfa7f91561ee6db89fff407986063866 Aug 04 12:59:58.935778 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:58.935678 2580 generic.go:356] "Generic (PLEG): container finished" podID="6849c27f-1f9b-4dab-9a85-9b74013521d9" containerID="5e59dce901db658a313a5c98ee731f3d1cdf5664cc0cdf2a52e15da83c4f26a3" exitCode=0 Aug 04 12:59:58.935778 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:58.935730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" event={"ID":"6849c27f-1f9b-4dab-9a85-9b74013521d9","Type":"ContainerDied","Data":"5e59dce901db658a313a5c98ee731f3d1cdf5664cc0cdf2a52e15da83c4f26a3"} Aug 04 12:59:58.935778 ip-10-0-132-96 kubenswrapper[2580]: I0804 12:59:58.935773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" event={"ID":"6849c27f-1f9b-4dab-9a85-9b74013521d9","Type":"ContainerStarted","Data":"0179a26c8877d9ed1297c352f5e01488dfa7f91561ee6db89fff407986063866"} Aug 04 13:00:00.074507 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.074479 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" Aug 04 13:00:00.135283 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.135240 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt"] Aug 04 13:00:00.135746 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.135728 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6849c27f-1f9b-4dab-9a85-9b74013521d9" containerName="revoke-keys" Aug 04 13:00:00.135802 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.135750 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6849c27f-1f9b-4dab-9a85-9b74013521d9" containerName="revoke-keys" Aug 04 13:00:00.135893 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.135880 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6849c27f-1f9b-4dab-9a85-9b74013521d9" containerName="revoke-keys" Aug 04 13:00:00.139096 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.139079 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54"] Aug 04 13:00:00.139225 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.139207 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" Aug 04 13:00:00.142315 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.142296 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" Aug 04 13:00:00.147317 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.147220 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54"] Aug 04 13:00:00.150134 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.149635 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt"] Aug 04 13:00:00.154756 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.154732 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" Aug 04 13:00:00.161679 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.161660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" Aug 04 13:00:00.177924 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.177896 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6849c27f-1f9b-4dab-9a85-9b74013521d9-maas-api-service-ca\") pod \"6849c27f-1f9b-4dab-9a85-9b74013521d9\" (UID: \"6849c27f-1f9b-4dab-9a85-9b74013521d9\") " Aug 04 13:00:00.178318 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.178291 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6849c27f-1f9b-4dab-9a85-9b74013521d9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6849c27f-1f9b-4dab-9a85-9b74013521d9" (UID: "6849c27f-1f9b-4dab-9a85-9b74013521d9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:00:00.279817 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.279780 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6849c27f-1f9b-4dab-9a85-9b74013521d9-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:00:00.310029 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.309993 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt"] Aug 04 13:00:00.312958 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:00:00.312929 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a3df01_05da_4286_b1e2_7cf828b0bf3b.slice/crio-27806666c4d98e30d3a0d7b6f19d7ddc2f3a907acad643ebc2e1ecc11001c538 WatchSource:0}: Error finding container 27806666c4d98e30d3a0d7b6f19d7ddc2f3a907acad643ebc2e1ecc11001c538: Status 404 returned error can't find the container with id 27806666c4d98e30d3a0d7b6f19d7ddc2f3a907acad643ebc2e1ecc11001c538 Aug 04 13:00:00.332363 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.332337 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54"] Aug 04 13:00:00.333301 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:00:00.333257 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9447ae7c_6676_4931_b9ce_cef7bf2d55b2.slice/crio-cf1e09bb09636c01e2be871e626e2f6561e9372f4055245dd8c0aa373ce4b672 WatchSource:0}: Error finding container cf1e09bb09636c01e2be871e626e2f6561e9372f4055245dd8c0aa373ce4b672: Status 404 returned error can't find the container with id cf1e09bb09636c01e2be871e626e2f6561e9372f4055245dd8c0aa373ce4b672 Aug 04 13:00:00.947627 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.947579 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" event={"ID":"9447ae7c-6676-4931-b9ce-cef7bf2d55b2","Type":"ContainerStarted","Data":"5d2ab17e9b7312fa6728a92e92bd319c1fb6657313728dd785cac56889e93ce2"} Aug 04 13:00:00.947813 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.947641 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" event={"ID":"9447ae7c-6676-4931-b9ce-cef7bf2d55b2","Type":"ContainerStarted","Data":"cf1e09bb09636c01e2be871e626e2f6561e9372f4055245dd8c0aa373ce4b672"} Aug 04 13:00:00.949178 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.949151 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" event={"ID":"33a3df01-05da-4286-b1e2-7cf828b0bf3b","Type":"ContainerStarted","Data":"00435eeea72aed4e054bd878fab220d8686b578a1e854e7823eb308add888a83"} Aug 04 13:00:00.949287 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.949193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" event={"ID":"33a3df01-05da-4286-b1e2-7cf828b0bf3b","Type":"ContainerStarted","Data":"27806666c4d98e30d3a0d7b6f19d7ddc2f3a907acad643ebc2e1ecc11001c538"} Aug 04 13:00:00.950360 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.950338 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" event={"ID":"6849c27f-1f9b-4dab-9a85-9b74013521d9","Type":"ContainerDied","Data":"0179a26c8877d9ed1297c352f5e01488dfa7f91561ee6db89fff407986063866"} Aug 04 13:00:00.950360 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.950350 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8" Aug 04 13:00:00.950480 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.950376 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0179a26c8877d9ed1297c352f5e01488dfa7f91561ee6db89fff407986063866" Aug 04 13:00:00.964706 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.964659 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" podStartSLOduration=0.96464584 podStartE2EDuration="964.64584ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:00:00.963637583 +0000 UTC m=+1961.200068966" watchObservedRunningTime="2026-08-04 13:00:00.96464584 +0000 UTC m=+1961.201077210" Aug 04 13:00:00.983728 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:00.983673 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" podStartSLOduration=0.983656864 podStartE2EDuration="983.656864ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:00:00.981999418 +0000 UTC m=+1961.218430790" watchObservedRunningTime="2026-08-04 13:00:00.983656864 +0000 UTC m=+1961.220088295" Aug 04 13:00:01.957007 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:01.956969 2580 generic.go:356] "Generic (PLEG): container finished" podID="9447ae7c-6676-4931-b9ce-cef7bf2d55b2" containerID="5d2ab17e9b7312fa6728a92e92bd319c1fb6657313728dd785cac56889e93ce2" exitCode=0 Aug 04 13:00:01.957503 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:01.957062 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" event={"ID":"9447ae7c-6676-4931-b9ce-cef7bf2d55b2","Type":"ContainerDied","Data":"5d2ab17e9b7312fa6728a92e92bd319c1fb6657313728dd785cac56889e93ce2"} Aug 04 13:00:01.958556 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:01.958531 2580 generic.go:356] "Generic (PLEG): container finished" podID="33a3df01-05da-4286-b1e2-7cf828b0bf3b" containerID="00435eeea72aed4e054bd878fab220d8686b578a1e854e7823eb308add888a83" exitCode=0 Aug 04 13:00:01.958646 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:01.958619 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" event={"ID":"33a3df01-05da-4286-b1e2-7cf828b0bf3b","Type":"ContainerDied","Data":"00435eeea72aed4e054bd878fab220d8686b578a1e854e7823eb308add888a83"} Aug 04 13:00:03.118684 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.118659 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" Aug 04 13:00:03.121395 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.121376 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" Aug 04 13:00:03.325603 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.325566 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54"] Aug 04 13:00:03.328280 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.328255 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54"] Aug 04 13:00:03.585232 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.585141 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8"] Aug 04 13:00:03.587954 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.587928 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fbddc9ad-rlrv8"] Aug 04 13:00:03.970044 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.969967 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" Aug 04 13:00:03.970044 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.970001 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf1e09bb09636c01e2be871e626e2f6561e9372f4055245dd8c0aa373ce4b672" Aug 04 13:00:03.971199 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.971175 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" event={"ID":"33a3df01-05da-4286-b1e2-7cf828b0bf3b","Type":"ContainerDied","Data":"27806666c4d98e30d3a0d7b6f19d7ddc2f3a907acad643ebc2e1ecc11001c538"} Aug 04 13:00:03.971280 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.971219 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27806666c4d98e30d3a0d7b6f19d7ddc2f3a907acad643ebc2e1ecc11001c538" Aug 04 13:00:03.971280 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:03.971185 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764140-p2hzt" Aug 04 13:00:03.986822 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:00:03.986793 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54\" is forbidden: User \"system:node:ip-10-0-132-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-132-96.ec2.internal' and this object" podUID="9447ae7c-6676-4931-b9ce-cef7bf2d55b2" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" Aug 04 13:00:03.988684 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:00:03.988663 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54\" is forbidden: User \"system:node:ip-10-0-132-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-132-96.ec2.internal' and this object" podUID="9447ae7c-6676-4931-b9ce-cef7bf2d55b2" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fbddc9ad-29764140-2fk54" Aug 04 13:00:04.277482 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:04.277404 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6849c27f-1f9b-4dab-9a85-9b74013521d9" path="/var/lib/kubelet/pods/6849c27f-1f9b-4dab-9a85-9b74013521d9/volumes" Aug 04 13:00:04.277809 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:04.277671 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9447ae7c-6676-4931-b9ce-cef7bf2d55b2" path="/var/lib/kubelet/pods/9447ae7c-6676-4931-b9ce-cef7bf2d55b2/volumes" Aug 04 13:00:06.197720 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:06.197677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:00:14.440468 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:14.440369 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:00:32.384926 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.384890 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4"] Aug 04 13:00:32.385411 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.385230 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9447ae7c-6676-4931-b9ce-cef7bf2d55b2" containerName="cleanup" Aug 04 13:00:32.385411 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.385241 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9447ae7c-6676-4931-b9ce-cef7bf2d55b2" containerName="cleanup" Aug 04 13:00:32.385411 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.385254 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a3df01-05da-4286-b1e2-7cf828b0bf3b" containerName="cleanup" Aug 04 13:00:32.385411 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.385260 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a3df01-05da-4286-b1e2-7cf828b0bf3b" containerName="cleanup" Aug 04 13:00:32.385411 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.385319 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="33a3df01-05da-4286-b1e2-7cf828b0bf3b" containerName="cleanup" Aug 04 13:00:32.385411 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.385327 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9447ae7c-6676-4931-b9ce-cef7bf2d55b2" containerName="cleanup" Aug 04 13:00:32.388675 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.388660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" Aug 04 13:00:32.391320 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.391302 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 13:00:32.391428 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.391359 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:00:32.397417 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.397377 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4"] Aug 04 13:00:32.448757 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.448706 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4\" (UID: \"6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" Aug 04 13:00:32.549438 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.549397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4\" (UID: \"6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" Aug 04 13:00:32.550085 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.550063 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4\" (UID: \"6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" Aug 04 13:00:32.700401 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.700312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" Aug 04 13:00:32.837344 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:32.837319 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4"] Aug 04 13:00:32.838701 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:00:32.838677 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6539ee88_00b0_40e7_ab9c_1fc1c7ab0c8f.slice/crio-e43a67ce41c8856d33d66d43dbb4b9685ca2f777830a37874448e8a4e860863b WatchSource:0}: Error finding container e43a67ce41c8856d33d66d43dbb4b9685ca2f777830a37874448e8a4e860863b: Status 404 returned error can't find the container with id e43a67ce41c8856d33d66d43dbb4b9685ca2f777830a37874448e8a4e860863b Aug 04 13:00:33.123546 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:33.123507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" event={"ID":"6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f","Type":"ContainerStarted","Data":"e43a67ce41c8856d33d66d43dbb4b9685ca2f777830a37874448e8a4e860863b"} Aug 04 13:00:34.131460 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:34.131423 2580 generic.go:356] "Generic (PLEG): container finished" podID="6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f" containerID="b1e23b7beda8ab914708bde118734deb838be6bb6a1cc2d45a5a38c11810de04" exitCode=0 Aug 04 13:00:34.131879 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:34.131482 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" event={"ID":"6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f","Type":"ContainerDied","Data":"b1e23b7beda8ab914708bde118734deb838be6bb6a1cc2d45a5a38c11810de04"} Aug 04 13:00:35.271087 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:35.271065 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" Aug 04 13:00:35.371920 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:35.371888 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f-maas-api-service-ca\") pod \"6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f\" (UID: \"6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f\") " Aug 04 13:00:35.372313 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:35.372283 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f" (UID: "6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:00:35.472760 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:35.472674 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:00:36.142649 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:36.142621 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" Aug 04 13:00:36.142816 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:36.142622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4" event={"ID":"6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f","Type":"ContainerDied","Data":"e43a67ce41c8856d33d66d43dbb4b9685ca2f777830a37874448e8a4e860863b"} Aug 04 13:00:36.142816 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:36.142746 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e43a67ce41c8856d33d66d43dbb4b9685ca2f777830a37874448e8a4e860863b" Aug 04 13:00:38.669664 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:38.669623 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4"] Aug 04 13:00:38.673331 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:38.673301 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f143cf24-zjgx4"] Aug 04 13:00:40.278301 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:40.278267 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f" path="/var/lib/kubelet/pods/6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f/volumes" Aug 04 13:00:41.491981 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:41.491943 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:00:50.104363 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:00:50.104329 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:01:07.656695 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.656646 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r"] Aug 04 13:01:07.657232 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.657063 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f" containerName="revoke-keys" Aug 04 13:01:07.657232 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.657077 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f" containerName="revoke-keys" Aug 04 13:01:07.657232 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.657129 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6539ee88-00b0-40e7-ab9c-1fc1c7ab0c8f" containerName="revoke-keys" Aug 04 13:01:07.659762 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.659747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" Aug 04 13:01:07.662143 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.662121 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 13:01:07.662253 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.662202 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:01:07.667850 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.667803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r"] Aug 04 13:01:07.758026 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.757993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ac6aaa-f7c0-497f-a573-7c95851458d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cb728584-zh87r\" (UID: \"d6ac6aaa-f7c0-497f-a573-7c95851458d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" Aug 04 13:01:07.858860 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.858812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ac6aaa-f7c0-497f-a573-7c95851458d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cb728584-zh87r\" (UID: \"d6ac6aaa-f7c0-497f-a573-7c95851458d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" Aug 04 13:01:07.859384 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.859366 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ac6aaa-f7c0-497f-a573-7c95851458d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cb728584-zh87r\" (UID: \"d6ac6aaa-f7c0-497f-a573-7c95851458d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" Aug 04 13:01:07.972622 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:07.972561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" Aug 04 13:01:08.126037 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:08.126010 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r"] Aug 04 13:01:08.127727 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:01:08.127698 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ac6aaa_f7c0_497f_a573_7c95851458d0.slice/crio-da10ae469b3494324a8fc14e7b72e85f201067594aa60d9ee0de32ee29816172 WatchSource:0}: Error finding container da10ae469b3494324a8fc14e7b72e85f201067594aa60d9ee0de32ee29816172: Status 404 returned error can't find the container with id da10ae469b3494324a8fc14e7b72e85f201067594aa60d9ee0de32ee29816172 Aug 04 13:01:08.319140 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:08.319111 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" event={"ID":"d6ac6aaa-f7c0-497f-a573-7c95851458d0","Type":"ContainerStarted","Data":"f1c62b5162292f8ac990fd64504a2fd23258dfc28671170a2581710ee44833ce"} Aug 04 13:01:08.319261 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:08.319163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" event={"ID":"d6ac6aaa-f7c0-497f-a573-7c95851458d0","Type":"ContainerStarted","Data":"da10ae469b3494324a8fc14e7b72e85f201067594aa60d9ee0de32ee29816172"} Aug 04 13:01:08.339847 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:08.339789 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" podStartSLOduration=1.339775908 podStartE2EDuration="1.339775908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:01:08.336997514 +0000 UTC m=+2028.573428885" watchObservedRunningTime="2026-08-04 13:01:08.339775908 +0000 UTC m=+2028.576207278" Aug 04 13:01:09.326409 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:09.326374 2580 generic.go:356] "Generic (PLEG): container finished" podID="d6ac6aaa-f7c0-497f-a573-7c95851458d0" containerID="f1c62b5162292f8ac990fd64504a2fd23258dfc28671170a2581710ee44833ce" exitCode=0 Aug 04 13:01:09.326806 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:09.326462 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" event={"ID":"d6ac6aaa-f7c0-497f-a573-7c95851458d0","Type":"ContainerDied","Data":"f1c62b5162292f8ac990fd64504a2fd23258dfc28671170a2581710ee44833ce"} Aug 04 13:01:10.462370 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:10.462346 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" Aug 04 13:01:10.582920 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:10.582813 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ac6aaa-f7c0-497f-a573-7c95851458d0-maas-api-service-ca\") pod \"d6ac6aaa-f7c0-497f-a573-7c95851458d0\" (UID: \"d6ac6aaa-f7c0-497f-a573-7c95851458d0\") " Aug 04 13:01:10.583170 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:10.583144 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ac6aaa-f7c0-497f-a573-7c95851458d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6ac6aaa-f7c0-497f-a573-7c95851458d0" (UID: "d6ac6aaa-f7c0-497f-a573-7c95851458d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:01:10.684188 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:10.684159 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ac6aaa-f7c0-497f-a573-7c95851458d0-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:01:11.338419 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:11.338388 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" Aug 04 13:01:11.338586 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:11.338426 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r" event={"ID":"d6ac6aaa-f7c0-497f-a573-7c95851458d0","Type":"ContainerDied","Data":"da10ae469b3494324a8fc14e7b72e85f201067594aa60d9ee0de32ee29816172"} Aug 04 13:01:11.338586 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:11.338476 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da10ae469b3494324a8fc14e7b72e85f201067594aa60d9ee0de32ee29816172" Aug 04 13:01:14.803759 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:14.803723 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r"] Aug 04 13:01:14.807246 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:14.807219 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cb728584-zh87r"] Aug 04 13:01:16.277072 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:16.277033 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6ac6aaa-f7c0-497f-a573-7c95851458d0" path="/var/lib/kubelet/pods/d6ac6aaa-f7c0-497f-a573-7c95851458d0/volumes" Aug 04 13:01:17.596544 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:17.596512 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:01:28.907443 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:01:28.907413 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:02:20.393790 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:02:20.393761 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:02:20.399940 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:02:20.399910 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:03:02.926769 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:03:02.926726 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:03:11.614027 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:03:11.613992 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:03:27.013144 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:03:27.013109 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:04:00.606800 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.606760 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf"] Aug 04 13:04:00.607407 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.607275 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6ac6aaa-f7c0-497f-a573-7c95851458d0" containerName="revoke-keys" Aug 04 13:04:00.607407 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.607296 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6ac6aaa-f7c0-497f-a573-7c95851458d0" containerName="revoke-keys" Aug 04 13:04:00.607407 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.607408 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6ac6aaa-f7c0-497f-a573-7c95851458d0" containerName="revoke-keys" Aug 04 13:04:00.610282 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.610262 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" Aug 04 13:04:00.613679 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.613657 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 13:04:00.613770 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.613754 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:04:00.619530 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.619509 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf"] Aug 04 13:04:00.652147 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.652115 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40df47d5-0c06-4e12-a0c2-fc73ceabcf3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf\" (UID: \"40df47d5-0c06-4e12-a0c2-fc73ceabcf3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" Aug 04 13:04:00.753088 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.753050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40df47d5-0c06-4e12-a0c2-fc73ceabcf3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf\" (UID: \"40df47d5-0c06-4e12-a0c2-fc73ceabcf3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" Aug 04 13:04:00.753699 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.753680 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40df47d5-0c06-4e12-a0c2-fc73ceabcf3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf\" (UID: \"40df47d5-0c06-4e12-a0c2-fc73ceabcf3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" Aug 04 13:04:00.921863 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:00.921755 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" Aug 04 13:04:01.062462 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:01.062433 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf"] Aug 04 13:04:01.063784 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:04:01.063758 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40df47d5_0c06_4e12_a0c2_fc73ceabcf3f.slice/crio-2fb40254546a60916cbaa7b9a874d423392420d661f6c2df169cf98c7b47fb37 WatchSource:0}: Error finding container 2fb40254546a60916cbaa7b9a874d423392420d661f6c2df169cf98c7b47fb37: Status 404 returned error can't find the container with id 2fb40254546a60916cbaa7b9a874d423392420d661f6c2df169cf98c7b47fb37 Aug 04 13:04:01.214495 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:01.214415 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" event={"ID":"40df47d5-0c06-4e12-a0c2-fc73ceabcf3f","Type":"ContainerStarted","Data":"2fb40254546a60916cbaa7b9a874d423392420d661f6c2df169cf98c7b47fb37"} Aug 04 13:04:02.221419 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:02.221382 2580 generic.go:356] "Generic (PLEG): container finished" podID="40df47d5-0c06-4e12-a0c2-fc73ceabcf3f" containerID="688956570ae86049f407afadb8fe5b68d988e844ec6538bb4baaf57825ad57fd" exitCode=0 Aug 04 13:04:02.221782 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:02.221463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" event={"ID":"40df47d5-0c06-4e12-a0c2-fc73ceabcf3f","Type":"ContainerDied","Data":"688956570ae86049f407afadb8fe5b68d988e844ec6538bb4baaf57825ad57fd"} Aug 04 13:04:03.359541 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:03.359515 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" Aug 04 13:04:03.375252 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:03.375221 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40df47d5-0c06-4e12-a0c2-fc73ceabcf3f-maas-api-service-ca\") pod \"40df47d5-0c06-4e12-a0c2-fc73ceabcf3f\" (UID: \"40df47d5-0c06-4e12-a0c2-fc73ceabcf3f\") " Aug 04 13:04:03.375570 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:03.375545 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40df47d5-0c06-4e12-a0c2-fc73ceabcf3f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40df47d5-0c06-4e12-a0c2-fc73ceabcf3f" (UID: "40df47d5-0c06-4e12-a0c2-fc73ceabcf3f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:04:03.476097 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:03.476053 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40df47d5-0c06-4e12-a0c2-fc73ceabcf3f-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:04:04.232340 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:04.232312 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" Aug 04 13:04:04.232509 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:04.232301 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf" event={"ID":"40df47d5-0c06-4e12-a0c2-fc73ceabcf3f","Type":"ContainerDied","Data":"2fb40254546a60916cbaa7b9a874d423392420d661f6c2df169cf98c7b47fb37"} Aug 04 13:04:04.232509 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:04.232435 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fb40254546a60916cbaa7b9a874d423392420d661f6c2df169cf98c7b47fb37" Aug 04 13:04:11.815071 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:11.815008 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf"] Aug 04 13:04:11.816624 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:11.816595 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d8882e5d-7jkxf"] Aug 04 13:04:12.276492 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:12.276406 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40df47d5-0c06-4e12-a0c2-fc73ceabcf3f" path="/var/lib/kubelet/pods/40df47d5-0c06-4e12-a0c2-fc73ceabcf3f/volumes" Aug 04 13:04:13.716264 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:13.716235 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:04:53.111814 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:04:53.111724 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:05:20.432661 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:05:20.432622 2580 scope.go:117] "RemoveContainer" containerID="a5adb207f801303fe8a00389d10ec3a9f1b3cb2690aacaecfa4b613fb594f90c" Aug 04 13:06:20.452964 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:06:20.452888 2580 scope.go:117] "RemoveContainer" containerID="5e59dce901db658a313a5c98ee731f3d1cdf5664cc0cdf2a52e15da83c4f26a3" Aug 04 13:06:20.461840 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:06:20.461807 2580 scope.go:117] "RemoveContainer" containerID="5d2ab17e9b7312fa6728a92e92bd319c1fb6657313728dd785cac56889e93ce2" Aug 04 13:06:39.285252 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:06:39.285210 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:06:46.104875 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:06:46.104823 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:07:20.430274 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:20.430245 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:07:20.437751 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:20.437726 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:07:20.484727 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:20.484688 2580 scope.go:117] "RemoveContainer" containerID="b1e23b7beda8ab914708bde118734deb838be6bb6a1cc2d45a5a38c11810de04" Aug 04 13:07:20.493302 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:20.493281 2580 scope.go:117] "RemoveContainer" containerID="f1c62b5162292f8ac990fd64504a2fd23258dfc28671170a2581710ee44833ce" Aug 04 13:07:27.903413 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:27.903380 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:07:43.720700 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:43.720605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:07:54.313553 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.313523 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5"] Aug 04 13:07:54.314043 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.313898 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40df47d5-0c06-4e12-a0c2-fc73ceabcf3f" containerName="revoke-keys" Aug 04 13:07:54.314043 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.313912 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="40df47d5-0c06-4e12-a0c2-fc73ceabcf3f" containerName="revoke-keys" Aug 04 13:07:54.314043 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.313971 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="40df47d5-0c06-4e12-a0c2-fc73ceabcf3f" containerName="revoke-keys" Aug 04 13:07:54.317114 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.317097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" Aug 04 13:07:54.320114 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.320088 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:07:54.320228 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.320117 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 13:07:54.326290 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.326254 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5"] Aug 04 13:07:54.420148 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.420123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fe4355f-56a7-4b3a-ac16-67f08746a579-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f758d543-vmxp5\" (UID: \"1fe4355f-56a7-4b3a-ac16-67f08746a579\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" Aug 04 13:07:54.521420 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.521391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fe4355f-56a7-4b3a-ac16-67f08746a579-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f758d543-vmxp5\" (UID: \"1fe4355f-56a7-4b3a-ac16-67f08746a579\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" Aug 04 13:07:54.522064 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.522038 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fe4355f-56a7-4b3a-ac16-67f08746a579-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f758d543-vmxp5\" (UID: \"1fe4355f-56a7-4b3a-ac16-67f08746a579\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" Aug 04 13:07:54.628233 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.628168 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" Aug 04 13:07:54.776342 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:54.776318 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5"] Aug 04 13:07:54.778306 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:07:54.778270 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe4355f_56a7_4b3a_ac16_67f08746a579.slice/crio-74a67c70d51c5e67abb2d5f6cff7a4ff4b53b44a5de1588077a77cffb5aeea2a WatchSource:0}: Error finding container 74a67c70d51c5e67abb2d5f6cff7a4ff4b53b44a5de1588077a77cffb5aeea2a: Status 404 returned error can't find the container with id 74a67c70d51c5e67abb2d5f6cff7a4ff4b53b44a5de1588077a77cffb5aeea2a Aug 04 13:07:55.434317 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:55.434282 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" event={"ID":"1fe4355f-56a7-4b3a-ac16-67f08746a579","Type":"ContainerStarted","Data":"7c3f15fdff8fb891f0dbd24ddd82551a6674b2be5bbc78d279dbfd4a210dc4a2"} Aug 04 13:07:55.434727 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:55.434332 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" event={"ID":"1fe4355f-56a7-4b3a-ac16-67f08746a579","Type":"ContainerStarted","Data":"74a67c70d51c5e67abb2d5f6cff7a4ff4b53b44a5de1588077a77cffb5aeea2a"} Aug 04 13:07:55.454482 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:55.454434 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" podStartSLOduration=1.454418921 podStartE2EDuration="1.454418921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:07:55.452248238 +0000 UTC m=+2435.688679609" watchObservedRunningTime="2026-08-04 13:07:55.454418921 +0000 UTC m=+2435.690850292" Aug 04 13:07:56.440906 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:56.440877 2580 generic.go:356] "Generic (PLEG): container finished" podID="1fe4355f-56a7-4b3a-ac16-67f08746a579" containerID="7c3f15fdff8fb891f0dbd24ddd82551a6674b2be5bbc78d279dbfd4a210dc4a2" exitCode=0 Aug 04 13:07:56.441312 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:56.440963 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" event={"ID":"1fe4355f-56a7-4b3a-ac16-67f08746a579","Type":"ContainerDied","Data":"7c3f15fdff8fb891f0dbd24ddd82551a6674b2be5bbc78d279dbfd4a210dc4a2"} Aug 04 13:07:57.577291 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:57.577264 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" Aug 04 13:07:57.649645 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:57.649618 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fe4355f-56a7-4b3a-ac16-67f08746a579-maas-api-service-ca\") pod \"1fe4355f-56a7-4b3a-ac16-67f08746a579\" (UID: \"1fe4355f-56a7-4b3a-ac16-67f08746a579\") " Aug 04 13:07:57.649994 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:57.649974 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fe4355f-56a7-4b3a-ac16-67f08746a579-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1fe4355f-56a7-4b3a-ac16-67f08746a579" (UID: "1fe4355f-56a7-4b3a-ac16-67f08746a579"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:07:57.750773 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:57.750721 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fe4355f-56a7-4b3a-ac16-67f08746a579-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:07:58.452408 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:58.452385 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" Aug 04 13:07:58.452408 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:58.452401 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5" event={"ID":"1fe4355f-56a7-4b3a-ac16-67f08746a579","Type":"ContainerDied","Data":"74a67c70d51c5e67abb2d5f6cff7a4ff4b53b44a5de1588077a77cffb5aeea2a"} Aug 04 13:07:58.452599 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:07:58.452449 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74a67c70d51c5e67abb2d5f6cff7a4ff4b53b44a5de1588077a77cffb5aeea2a" Aug 04 13:08:07.610009 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:08:07.609974 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:08:10.284572 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:08:10.284538 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5"] Aug 04 13:08:10.284951 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:08:10.284586 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f758d543-vmxp5"] Aug 04 13:08:12.277191 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:08:12.277156 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fe4355f-56a7-4b3a-ac16-67f08746a579" path="/var/lib/kubelet/pods/1fe4355f-56a7-4b3a-ac16-67f08746a579/volumes" Aug 04 13:08:13.500759 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:08:13.500726 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:08:45.011313 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:08:45.011281 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:08:56.730368 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:08:56.730336 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:09:01.708658 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:09:01.708620 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:09:10.265593 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:09:10.265554 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:10:02.103753 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:02.103720 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:10:20.532260 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:20.532232 2580 scope.go:117] "RemoveContainer" containerID="688956570ae86049f407afadb8fe5b68d988e844ec6538bb4baaf57825ad57fd" Aug 04 13:10:27.689345 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.689314 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd"] Aug 04 13:10:27.689825 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.689804 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe4355f-56a7-4b3a-ac16-67f08746a579" containerName="revoke-keys" Aug 04 13:10:27.689924 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.689845 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe4355f-56a7-4b3a-ac16-67f08746a579" containerName="revoke-keys" Aug 04 13:10:27.689985 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.689954 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fe4355f-56a7-4b3a-ac16-67f08746a579" containerName="revoke-keys" Aug 04 13:10:27.692897 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.692879 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.696556 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.696529 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 13:10:27.696660 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.696590 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3a8b01-serving-cert\"" Aug 04 13:10:27.696660 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.696595 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 13:10:27.699859 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.699814 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p76m5\"" Aug 04 13:10:27.707289 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.707268 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd"] Aug 04 13:10:27.775051 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.775028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tgwl\" (UniqueName: \"kubernetes.io/projected/b62d21ed-304f-40a1-baa4-12a9d453bdc8-kube-api-access-2tgwl\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.775156 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.775065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62d21ed-304f-40a1-baa4-12a9d453bdc8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.775156 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.775086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b62d21ed-304f-40a1-baa4-12a9d453bdc8-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.876400 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.876369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tgwl\" (UniqueName: \"kubernetes.io/projected/b62d21ed-304f-40a1-baa4-12a9d453bdc8-kube-api-access-2tgwl\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.876559 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.876410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62d21ed-304f-40a1-baa4-12a9d453bdc8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.876559 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.876437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b62d21ed-304f-40a1-baa4-12a9d453bdc8-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.877005 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.876987 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62d21ed-304f-40a1-baa4-12a9d453bdc8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.878937 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.878913 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b62d21ed-304f-40a1-baa4-12a9d453bdc8-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:27.888046 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:27.888023 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tgwl\" (UniqueName: \"kubernetes.io/projected/b62d21ed-304f-40a1-baa4-12a9d453bdc8-kube-api-access-2tgwl\") pod \"maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:28.003189 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:28.003126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:28.138407 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:28.138379 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd"] Aug 04 13:10:28.139823 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:10:28.139795 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62d21ed_304f_40a1_baa4_12a9d453bdc8.slice/crio-5b3fe5c0ace85015f4fee3b77879c6d3dd1cc2e6f0ca33e65554c8479b44e659 WatchSource:0}: Error finding container 5b3fe5c0ace85015f4fee3b77879c6d3dd1cc2e6f0ca33e65554c8479b44e659: Status 404 returned error can't find the container with id 5b3fe5c0ace85015f4fee3b77879c6d3dd1cc2e6f0ca33e65554c8479b44e659 Aug 04 13:10:28.141117 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:28.141077 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 13:10:28.207765 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:28.207737 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" event={"ID":"b62d21ed-304f-40a1-baa4-12a9d453bdc8","Type":"ContainerStarted","Data":"5b3fe5c0ace85015f4fee3b77879c6d3dd1cc2e6f0ca33e65554c8479b44e659"} Aug 04 13:10:30.604581 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:30.604541 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:10:31.224708 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:31.224676 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" event={"ID":"b62d21ed-304f-40a1-baa4-12a9d453bdc8","Type":"ContainerStarted","Data":"7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da"} Aug 04 13:10:31.224950 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:31.224740 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:10:31.245288 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:31.245239 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" podStartSLOduration=2.138043632 podStartE2EDuration="4.245224334s" totalImagesPullingTime="2.107180702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:10:31.243188751 +0000 UTC m=+2591.479620127" watchObservedRunningTime="2026-08-04 13:10:31.245224334 +0000 UTC m=+2591.481655728" Aug 04 13:10:37.236031 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:10:37.236005 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:11:07.219643 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:11:07.219561 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:11:14.298959 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:11:14.298925 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:11:26.301527 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:11:26.301489 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:12:02.333556 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.333513 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5"] Aug 04 13:12:02.335898 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.335879 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" Aug 04 13:12:02.338630 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.338602 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:12:02.345681 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.345593 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5"] Aug 04 13:12:02.374673 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.374635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50abbf64-8988-4b0a-a63f-68b736020087-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5\" (UID: \"50abbf64-8988-4b0a-a63f-68b736020087\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" Aug 04 13:12:02.475251 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.475218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50abbf64-8988-4b0a-a63f-68b736020087-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5\" (UID: \"50abbf64-8988-4b0a-a63f-68b736020087\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" Aug 04 13:12:02.475821 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.475801 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50abbf64-8988-4b0a-a63f-68b736020087-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5\" (UID: \"50abbf64-8988-4b0a-a63f-68b736020087\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" Aug 04 13:12:02.647648 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.647547 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" Aug 04 13:12:02.790446 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:02.790421 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5"] Aug 04 13:12:02.792812 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:12:02.792785 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50abbf64_8988_4b0a_a63f_68b736020087.slice/crio-bf5c05b0bfb0bd25d4a8f8960eb815450964cc2401622869c771eb3d71f5afad WatchSource:0}: Error finding container bf5c05b0bfb0bd25d4a8f8960eb815450964cc2401622869c771eb3d71f5afad: Status 404 returned error can't find the container with id bf5c05b0bfb0bd25d4a8f8960eb815450964cc2401622869c771eb3d71f5afad Aug 04 13:12:03.683898 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:03.683864 2580 generic.go:356] "Generic (PLEG): container finished" podID="50abbf64-8988-4b0a-a63f-68b736020087" containerID="7cacf5ca0760acd6f85691429847dd33145f55d09d8624f82a8dd158996b9df8" exitCode=0 Aug 04 13:12:03.684277 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:03.683916 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" event={"ID":"50abbf64-8988-4b0a-a63f-68b736020087","Type":"ContainerDied","Data":"7cacf5ca0760acd6f85691429847dd33145f55d09d8624f82a8dd158996b9df8"} Aug 04 13:12:03.684277 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:03.683953 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" event={"ID":"50abbf64-8988-4b0a-a63f-68b736020087","Type":"ContainerStarted","Data":"bf5c05b0bfb0bd25d4a8f8960eb815450964cc2401622869c771eb3d71f5afad"} Aug 04 13:12:04.717460 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:04.717425 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:12:04.824393 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:04.824367 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" Aug 04 13:12:04.898445 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:04.898409 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50abbf64-8988-4b0a-a63f-68b736020087-maas-api-service-ca\") pod \"50abbf64-8988-4b0a-a63f-68b736020087\" (UID: \"50abbf64-8988-4b0a-a63f-68b736020087\") " Aug 04 13:12:04.898781 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:04.898759 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50abbf64-8988-4b0a-a63f-68b736020087-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50abbf64-8988-4b0a-a63f-68b736020087" (UID: "50abbf64-8988-4b0a-a63f-68b736020087"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:12:04.999396 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:04.999303 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50abbf64-8988-4b0a-a63f-68b736020087-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:12:05.696771 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:05.696737 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" Aug 04 13:12:05.696971 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:05.696737 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5" event={"ID":"50abbf64-8988-4b0a-a63f-68b736020087","Type":"ContainerDied","Data":"bf5c05b0bfb0bd25d4a8f8960eb815450964cc2401622869c771eb3d71f5afad"} Aug 04 13:12:05.696971 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:05.696906 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf5c05b0bfb0bd25d4a8f8960eb815450964cc2401622869c771eb3d71f5afad" Aug 04 13:12:13.268909 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:13.268810 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5"] Aug 04 13:12:13.272523 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:13.272498 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1c6080-8pkb5"] Aug 04 13:12:14.277129 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:14.277094 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50abbf64-8988-4b0a-a63f-68b736020087" path="/var/lib/kubelet/pods/50abbf64-8988-4b0a-a63f-68b736020087/volumes" Aug 04 13:12:15.309772 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:15.309729 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:12:20.466237 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.466196 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:12:20.473219 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.473191 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:12:20.791172 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.791070 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g"] Aug 04 13:12:20.791543 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.791522 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50abbf64-8988-4b0a-a63f-68b736020087" containerName="revoke-keys" Aug 04 13:12:20.791640 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.791546 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="50abbf64-8988-4b0a-a63f-68b736020087" containerName="revoke-keys" Aug 04 13:12:20.791702 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.791661 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="50abbf64-8988-4b0a-a63f-68b736020087" containerName="revoke-keys" Aug 04 13:12:20.794584 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.794559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" Aug 04 13:12:20.797211 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.797189 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:12:20.803714 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.803687 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g"] Aug 04 13:12:20.936282 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:20.936223 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bbb74d-7329-48ad-bfa7-5eb54252c30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g\" (UID: \"e2bbb74d-7329-48ad-bfa7-5eb54252c30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" Aug 04 13:12:21.037841 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:21.037803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bbb74d-7329-48ad-bfa7-5eb54252c30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g\" (UID: \"e2bbb74d-7329-48ad-bfa7-5eb54252c30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" Aug 04 13:12:21.038437 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:21.038415 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bbb74d-7329-48ad-bfa7-5eb54252c30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g\" (UID: \"e2bbb74d-7329-48ad-bfa7-5eb54252c30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" Aug 04 13:12:21.106335 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:21.106293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" Aug 04 13:12:21.242330 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:21.242304 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g"] Aug 04 13:12:21.244189 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:12:21.244162 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2bbb74d_7329_48ad_bfa7_5eb54252c30a.slice/crio-8301ad37593c80984a81f3acc2cafa211b1e64c43f21aa5d7fc59bc6f50f50fd WatchSource:0}: Error finding container 8301ad37593c80984a81f3acc2cafa211b1e64c43f21aa5d7fc59bc6f50f50fd: Status 404 returned error can't find the container with id 8301ad37593c80984a81f3acc2cafa211b1e64c43f21aa5d7fc59bc6f50f50fd Aug 04 13:12:21.779768 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:21.779670 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" event={"ID":"e2bbb74d-7329-48ad-bfa7-5eb54252c30a","Type":"ContainerStarted","Data":"c19ac9df223e91031b5c3303927fe684bd638ea6f6aea93c253f29e25025a6c3"} Aug 04 13:12:21.779768 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:21.779728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" event={"ID":"e2bbb74d-7329-48ad-bfa7-5eb54252c30a","Type":"ContainerStarted","Data":"8301ad37593c80984a81f3acc2cafa211b1e64c43f21aa5d7fc59bc6f50f50fd"} Aug 04 13:12:21.798853 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:21.798775 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" podStartSLOduration=1.7987585259999999 podStartE2EDuration="1.798758526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:12:21.7970969 +0000 UTC m=+2702.033528252" watchObservedRunningTime="2026-08-04 13:12:21.798758526 +0000 UTC m=+2702.035189895" Aug 04 13:12:22.786568 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:22.786535 2580 generic.go:356] "Generic (PLEG): container finished" podID="e2bbb74d-7329-48ad-bfa7-5eb54252c30a" containerID="c19ac9df223e91031b5c3303927fe684bd638ea6f6aea93c253f29e25025a6c3" exitCode=0 Aug 04 13:12:22.787005 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:22.786584 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" event={"ID":"e2bbb74d-7329-48ad-bfa7-5eb54252c30a","Type":"ContainerDied","Data":"c19ac9df223e91031b5c3303927fe684bd638ea6f6aea93c253f29e25025a6c3"} Aug 04 13:12:23.926528 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:23.926501 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" Aug 04 13:12:24.063085 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:24.063053 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bbb74d-7329-48ad-bfa7-5eb54252c30a-maas-api-service-ca\") pod \"e2bbb74d-7329-48ad-bfa7-5eb54252c30a\" (UID: \"e2bbb74d-7329-48ad-bfa7-5eb54252c30a\") " Aug 04 13:12:24.063420 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:24.063398 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2bbb74d-7329-48ad-bfa7-5eb54252c30a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2bbb74d-7329-48ad-bfa7-5eb54252c30a" (UID: "e2bbb74d-7329-48ad-bfa7-5eb54252c30a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:12:24.164058 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:24.164026 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bbb74d-7329-48ad-bfa7-5eb54252c30a-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:12:24.798922 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:24.798881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" event={"ID":"e2bbb74d-7329-48ad-bfa7-5eb54252c30a","Type":"ContainerDied","Data":"8301ad37593c80984a81f3acc2cafa211b1e64c43f21aa5d7fc59bc6f50f50fd"} Aug 04 13:12:24.799093 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:24.798934 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8301ad37593c80984a81f3acc2cafa211b1e64c43f21aa5d7fc59bc6f50f50fd" Aug 04 13:12:24.799093 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:24.798893 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g" Aug 04 13:12:31.306824 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.306789 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd"] Aug 04 13:12:31.307260 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.307031 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" podUID="b62d21ed-304f-40a1-baa4-12a9d453bdc8" containerName="maas-api" containerID="cri-o://7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da" gracePeriod=30 Aug 04 13:12:31.526412 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.526376 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g"] Aug 04 13:12:31.529529 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.529503 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a8b01-csz9g"] Aug 04 13:12:31.556302 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.556280 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:12:31.629180 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.629078 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62d21ed-304f-40a1-baa4-12a9d453bdc8-openshift-service-ca\") pod \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " Aug 04 13:12:31.629180 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.629166 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b62d21ed-304f-40a1-baa4-12a9d453bdc8-maas-api-tls\") pod \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " Aug 04 13:12:31.629428 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.629207 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tgwl\" (UniqueName: \"kubernetes.io/projected/b62d21ed-304f-40a1-baa4-12a9d453bdc8-kube-api-access-2tgwl\") pod \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\" (UID: \"b62d21ed-304f-40a1-baa4-12a9d453bdc8\") " Aug 04 13:12:31.629544 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.629517 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b62d21ed-304f-40a1-baa4-12a9d453bdc8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b62d21ed-304f-40a1-baa4-12a9d453bdc8" (UID: "b62d21ed-304f-40a1-baa4-12a9d453bdc8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:12:31.631635 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.631605 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62d21ed-304f-40a1-baa4-12a9d453bdc8-kube-api-access-2tgwl" (OuterVolumeSpecName: "kube-api-access-2tgwl") pod "b62d21ed-304f-40a1-baa4-12a9d453bdc8" (UID: "b62d21ed-304f-40a1-baa4-12a9d453bdc8"). InnerVolumeSpecName "kube-api-access-2tgwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:12:31.631635 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.631618 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b62d21ed-304f-40a1-baa4-12a9d453bdc8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b62d21ed-304f-40a1-baa4-12a9d453bdc8" (UID: "b62d21ed-304f-40a1-baa4-12a9d453bdc8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 13:12:31.730442 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.730398 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tgwl\" (UniqueName: \"kubernetes.io/projected/b62d21ed-304f-40a1-baa4-12a9d453bdc8-kube-api-access-2tgwl\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:12:31.730442 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.730435 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62d21ed-304f-40a1-baa4-12a9d453bdc8-openshift-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:12:31.730442 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.730446 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b62d21ed-304f-40a1-baa4-12a9d453bdc8-maas-api-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:12:31.837717 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.837684 2580 generic.go:356] "Generic (PLEG): container finished" podID="b62d21ed-304f-40a1-baa4-12a9d453bdc8" containerID="7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da" exitCode=0 Aug 04 13:12:31.837945 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.837733 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" Aug 04 13:12:31.837945 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.837773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" event={"ID":"b62d21ed-304f-40a1-baa4-12a9d453bdc8","Type":"ContainerDied","Data":"7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da"} Aug 04 13:12:31.837945 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.837853 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd" event={"ID":"b62d21ed-304f-40a1-baa4-12a9d453bdc8","Type":"ContainerDied","Data":"5b3fe5c0ace85015f4fee3b77879c6d3dd1cc2e6f0ca33e65554c8479b44e659"} Aug 04 13:12:31.837945 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.837878 2580 scope.go:117] "RemoveContainer" containerID="7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da" Aug 04 13:12:31.849820 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.849796 2580 scope.go:117] "RemoveContainer" containerID="7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da" Aug 04 13:12:31.850208 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:12:31.850188 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da\": container with ID starting with 7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da not found: ID does not exist" containerID="7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da" Aug 04 13:12:31.850295 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.850214 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da"} err="failed to get container status \"7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da\": rpc error: code = NotFound desc = could not find container \"7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da\": container with ID starting with 7c384d23d5814434c1d407d62784bea6ab0614331d529b280cfbf127261db9da not found: ID does not exist" Aug 04 13:12:31.862808 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.862780 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd"] Aug 04 13:12:31.866571 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:31.866545 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a8b01-6ddf49c499-tp8fd"] Aug 04 13:12:32.276668 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:32.276634 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b62d21ed-304f-40a1-baa4-12a9d453bdc8" path="/var/lib/kubelet/pods/b62d21ed-304f-40a1-baa4-12a9d453bdc8/volumes" Aug 04 13:12:32.277042 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:32.277026 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2bbb74d-7329-48ad-bfa7-5eb54252c30a" path="/var/lib/kubelet/pods/e2bbb74d-7329-48ad-bfa7-5eb54252c30a/volumes" Aug 04 13:12:33.506860 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:33.502726 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:12:43.241460 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.241425 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6"] Aug 04 13:12:43.241910 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.241759 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2bbb74d-7329-48ad-bfa7-5eb54252c30a" containerName="revoke-keys" Aug 04 13:12:43.241910 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.241771 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2bbb74d-7329-48ad-bfa7-5eb54252c30a" containerName="revoke-keys" Aug 04 13:12:43.241910 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.241789 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b62d21ed-304f-40a1-baa4-12a9d453bdc8" containerName="maas-api" Aug 04 13:12:43.241910 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.241795 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b62d21ed-304f-40a1-baa4-12a9d453bdc8" containerName="maas-api" Aug 04 13:12:43.241910 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.241896 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2bbb74d-7329-48ad-bfa7-5eb54252c30a" containerName="revoke-keys" Aug 04 13:12:43.241910 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.241906 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b62d21ed-304f-40a1-baa4-12a9d453bdc8" containerName="maas-api" Aug 04 13:12:43.244839 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.244805 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.247665 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.247644 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p76m5\"" Aug 04 13:12:43.247784 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.247672 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-0c5286-serving-cert\"" Aug 04 13:12:43.247784 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.247682 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 13:12:43.248700 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.248685 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 13:12:43.257916 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.257887 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6"] Aug 04 13:12:43.432244 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.432192 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848ab4d4-e393-4bd3-801c-9d1a239f7876-maas-api-tls\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.432244 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.432249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqmgr\" (UniqueName: \"kubernetes.io/projected/848ab4d4-e393-4bd3-801c-9d1a239f7876-kube-api-access-bqmgr\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.432472 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.432352 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848ab4d4-e393-4bd3-801c-9d1a239f7876-openshift-service-ca\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.533399 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.533291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848ab4d4-e393-4bd3-801c-9d1a239f7876-maas-api-tls\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.533399 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.533355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqmgr\" (UniqueName: \"kubernetes.io/projected/848ab4d4-e393-4bd3-801c-9d1a239f7876-kube-api-access-bqmgr\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.533399 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.533400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848ab4d4-e393-4bd3-801c-9d1a239f7876-openshift-service-ca\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.534100 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.534076 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848ab4d4-e393-4bd3-801c-9d1a239f7876-openshift-service-ca\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.536014 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.535986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848ab4d4-e393-4bd3-801c-9d1a239f7876-maas-api-tls\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.543740 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.543720 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqmgr\" (UniqueName: \"kubernetes.io/projected/848ab4d4-e393-4bd3-801c-9d1a239f7876-kube-api-access-bqmgr\") pod \"maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.555573 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.555551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:43.702018 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.701992 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6"] Aug 04 13:12:43.704155 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:12:43.704126 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848ab4d4_e393_4bd3_801c_9d1a239f7876.slice/crio-cd121804299e734aa160da80d464825fbcafe220e4bf6d88e3eb84b639082787 WatchSource:0}: Error finding container cd121804299e734aa160da80d464825fbcafe220e4bf6d88e3eb84b639082787: Status 404 returned error can't find the container with id cd121804299e734aa160da80d464825fbcafe220e4bf6d88e3eb84b639082787 Aug 04 13:12:43.901582 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:43.901537 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" event={"ID":"848ab4d4-e393-4bd3-801c-9d1a239f7876","Type":"ContainerStarted","Data":"cd121804299e734aa160da80d464825fbcafe220e4bf6d88e3eb84b639082787"} Aug 04 13:12:44.908597 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:44.908507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" event={"ID":"848ab4d4-e393-4bd3-801c-9d1a239f7876","Type":"ContainerStarted","Data":"ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4"} Aug 04 13:12:44.909021 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:44.908634 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:12:44.929387 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:44.929338 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" podStartSLOduration=1.080389474 podStartE2EDuration="1.929322993s" totalImagesPullingTime="848.933519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:12:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:12:44.928126545 +0000 UTC m=+2725.164557916" watchObservedRunningTime="2026-08-04 13:12:44.929322993 +0000 UTC m=+2725.165754360" Aug 04 13:12:45.803400 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:45.803365 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:12:50.918914 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:12:50.918885 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:13:12.470450 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.470416 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv"] Aug 04 13:13:12.472745 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.472726 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:12.476278 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.476256 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c1a14d-serving-cert\"" Aug 04 13:13:12.485954 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.485927 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv"] Aug 04 13:13:12.591048 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.591012 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:12.591225 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.591055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9d36f4-7b1d-497f-97e0-608434cdd168-openshift-service-ca\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:12.591225 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.591084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp6kb\" (UniqueName: \"kubernetes.io/projected/8d9d36f4-7b1d-497f-97e0-608434cdd168-kube-api-access-fp6kb\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:12.691823 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.691792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:12.692002 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.691863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9d36f4-7b1d-497f-97e0-608434cdd168-openshift-service-ca\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:12.692002 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.691891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp6kb\" (UniqueName: \"kubernetes.io/projected/8d9d36f4-7b1d-497f-97e0-608434cdd168-kube-api-access-fp6kb\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:12.692002 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:13:12.691964 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-serving-cert: secret "maas-api-e2e-api-b-c1a14d-serving-cert" not found Aug 04 13:13:12.692138 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:13:12.692039 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls podName:8d9d36f4-7b1d-497f-97e0-608434cdd168 nodeName:}" failed. No retries permitted until 2026-08-04 13:13:13.192015452 +0000 UTC m=+2753.428446801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls") pod "maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" (UID: "8d9d36f4-7b1d-497f-97e0-608434cdd168") : secret "maas-api-e2e-api-b-c1a14d-serving-cert" not found Aug 04 13:13:12.692515 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.692497 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9d36f4-7b1d-497f-97e0-608434cdd168-openshift-service-ca\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:12.704095 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:12.704069 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp6kb\" (UniqueName: \"kubernetes.io/projected/8d9d36f4-7b1d-497f-97e0-608434cdd168-kube-api-access-fp6kb\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:13.196746 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:13.196707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:13.199546 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:13.199512 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls\") pod \"maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:13.382438 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:13.382398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:13.523761 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:13.523734 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv"] Aug 04 13:13:13.525334 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:13:13.525307 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9d36f4_7b1d_497f_97e0_608434cdd168.slice/crio-f2ad62892980187fb313a0b2a3e976b3a5d943f6de2d0b046def85cf3de05519 WatchSource:0}: Error finding container f2ad62892980187fb313a0b2a3e976b3a5d943f6de2d0b046def85cf3de05519: Status 404 returned error can't find the container with id f2ad62892980187fb313a0b2a3e976b3a5d943f6de2d0b046def85cf3de05519 Aug 04 13:13:14.061749 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:14.061719 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" event={"ID":"8d9d36f4-7b1d-497f-97e0-608434cdd168","Type":"ContainerStarted","Data":"f2ad62892980187fb313a0b2a3e976b3a5d943f6de2d0b046def85cf3de05519"} Aug 04 13:13:15.068364 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:15.068322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" event={"ID":"8d9d36f4-7b1d-497f-97e0-608434cdd168","Type":"ContainerStarted","Data":"56b7f954943c6b69ea5c0b9abb359ed1828c108f64df13964203aa3cc9afce89"} Aug 04 13:13:15.068809 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:15.068389 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:15.093425 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:15.093374 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" podStartSLOduration=2.601701213 podStartE2EDuration="3.093357228s" totalImagesPullingTime="491.656015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:13:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:13:15.092037961 +0000 UTC m=+2755.328469330" watchObservedRunningTime="2026-08-04 13:13:15.093357228 +0000 UTC m=+2755.329788598" Aug 04 13:13:16.717673 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:16.717632 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:13:21.078879 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:21.078853 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:13:39.390695 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.390664 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd"] Aug 04 13:13:39.393109 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.393094 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" Aug 04 13:13:39.396100 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.396080 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:13:39.405267 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.405245 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd"] Aug 04 13:13:39.411707 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.411677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bd2178-dda5-4595-b770-65444440257b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd\" (UID: \"81bd2178-dda5-4595-b770-65444440257b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" Aug 04 13:13:39.513525 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.513317 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bd2178-dda5-4595-b770-65444440257b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd\" (UID: \"81bd2178-dda5-4595-b770-65444440257b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" Aug 04 13:13:39.518684 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.514362 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bd2178-dda5-4595-b770-65444440257b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd\" (UID: \"81bd2178-dda5-4595-b770-65444440257b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" Aug 04 13:13:39.702728 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.702637 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" Aug 04 13:13:39.854041 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:39.854015 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd"] Aug 04 13:13:39.855825 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:13:39.855800 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81bd2178_dda5_4595_b770_65444440257b.slice/crio-1ddc9965ff9d412071358709141b8d10d5342b77c31c91cde3acfe07d9a85c74 WatchSource:0}: Error finding container 1ddc9965ff9d412071358709141b8d10d5342b77c31c91cde3acfe07d9a85c74: Status 404 returned error can't find the container with id 1ddc9965ff9d412071358709141b8d10d5342b77c31c91cde3acfe07d9a85c74 Aug 04 13:13:40.199849 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:40.199784 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" event={"ID":"81bd2178-dda5-4595-b770-65444440257b","Type":"ContainerStarted","Data":"58d33286a21c7ccbf56bf97f9e85b5ace5882df848cc53111315ebfacba3904f"} Aug 04 13:13:40.200029 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:40.199872 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" event={"ID":"81bd2178-dda5-4595-b770-65444440257b","Type":"ContainerStarted","Data":"1ddc9965ff9d412071358709141b8d10d5342b77c31c91cde3acfe07d9a85c74"} Aug 04 13:13:40.218756 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:40.218706 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" podStartSLOduration=1.218686571 podStartE2EDuration="1.218686571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:13:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:13:40.217520423 +0000 UTC m=+2780.453951791" watchObservedRunningTime="2026-08-04 13:13:40.218686571 +0000 UTC m=+2780.455117943" Aug 04 13:13:41.206235 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:41.206205 2580 generic.go:356] "Generic (PLEG): container finished" podID="81bd2178-dda5-4595-b770-65444440257b" containerID="58d33286a21c7ccbf56bf97f9e85b5ace5882df848cc53111315ebfacba3904f" exitCode=0 Aug 04 13:13:41.206641 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:41.206293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" event={"ID":"81bd2178-dda5-4595-b770-65444440257b","Type":"ContainerDied","Data":"58d33286a21c7ccbf56bf97f9e85b5ace5882df848cc53111315ebfacba3904f"} Aug 04 13:13:42.346992 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:42.346970 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" Aug 04 13:13:42.438423 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:42.438392 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bd2178-dda5-4595-b770-65444440257b-maas-api-service-ca\") pod \"81bd2178-dda5-4595-b770-65444440257b\" (UID: \"81bd2178-dda5-4595-b770-65444440257b\") " Aug 04 13:13:42.438737 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:42.438713 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81bd2178-dda5-4595-b770-65444440257b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81bd2178-dda5-4595-b770-65444440257b" (UID: "81bd2178-dda5-4595-b770-65444440257b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:13:42.539259 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:42.539177 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bd2178-dda5-4595-b770-65444440257b-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:13:43.217602 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:43.217568 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" Aug 04 13:13:43.217769 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:43.217610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd" event={"ID":"81bd2178-dda5-4595-b770-65444440257b","Type":"ContainerDied","Data":"1ddc9965ff9d412071358709141b8d10d5342b77c31c91cde3acfe07d9a85c74"} Aug 04 13:13:43.217769 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:43.217657 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ddc9965ff9d412071358709141b8d10d5342b77c31c91cde3acfe07d9a85c74" Aug 04 13:13:52.224052 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.224014 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6"] Aug 04 13:13:52.224548 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.224253 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" podUID="848ab4d4-e393-4bd3-801c-9d1a239f7876" containerName="maas-api" containerID="cri-o://ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4" gracePeriod=30 Aug 04 13:13:52.470021 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.469997 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:13:52.490102 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.490015 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd"] Aug 04 13:13:52.499700 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.499667 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0c5286-zk9gd"] Aug 04 13:13:52.521668 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.521639 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqmgr\" (UniqueName: \"kubernetes.io/projected/848ab4d4-e393-4bd3-801c-9d1a239f7876-kube-api-access-bqmgr\") pod \"848ab4d4-e393-4bd3-801c-9d1a239f7876\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " Aug 04 13:13:52.521990 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.521680 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848ab4d4-e393-4bd3-801c-9d1a239f7876-maas-api-tls\") pod \"848ab4d4-e393-4bd3-801c-9d1a239f7876\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " Aug 04 13:13:52.521990 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.521719 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848ab4d4-e393-4bd3-801c-9d1a239f7876-openshift-service-ca\") pod \"848ab4d4-e393-4bd3-801c-9d1a239f7876\" (UID: \"848ab4d4-e393-4bd3-801c-9d1a239f7876\") " Aug 04 13:13:52.522228 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.522199 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/848ab4d4-e393-4bd3-801c-9d1a239f7876-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "848ab4d4-e393-4bd3-801c-9d1a239f7876" (UID: "848ab4d4-e393-4bd3-801c-9d1a239f7876"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:13:52.524096 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.524068 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/848ab4d4-e393-4bd3-801c-9d1a239f7876-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "848ab4d4-e393-4bd3-801c-9d1a239f7876" (UID: "848ab4d4-e393-4bd3-801c-9d1a239f7876"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 13:13:52.524242 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.524227 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/848ab4d4-e393-4bd3-801c-9d1a239f7876-kube-api-access-bqmgr" (OuterVolumeSpecName: "kube-api-access-bqmgr") pod "848ab4d4-e393-4bd3-801c-9d1a239f7876" (UID: "848ab4d4-e393-4bd3-801c-9d1a239f7876"). InnerVolumeSpecName "kube-api-access-bqmgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:13:52.622556 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.622524 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqmgr\" (UniqueName: \"kubernetes.io/projected/848ab4d4-e393-4bd3-801c-9d1a239f7876-kube-api-access-bqmgr\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:13:52.622556 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.622556 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848ab4d4-e393-4bd3-801c-9d1a239f7876-maas-api-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:13:52.622756 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:52.622571 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848ab4d4-e393-4bd3-801c-9d1a239f7876-openshift-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:13:53.272654 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.272615 2580 generic.go:356] "Generic (PLEG): container finished" podID="848ab4d4-e393-4bd3-801c-9d1a239f7876" containerID="ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4" exitCode=0 Aug 04 13:13:53.273130 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.272661 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" Aug 04 13:13:53.273130 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.272708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" event={"ID":"848ab4d4-e393-4bd3-801c-9d1a239f7876","Type":"ContainerDied","Data":"ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4"} Aug 04 13:13:53.273130 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.272753 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6" event={"ID":"848ab4d4-e393-4bd3-801c-9d1a239f7876","Type":"ContainerDied","Data":"cd121804299e734aa160da80d464825fbcafe220e4bf6d88e3eb84b639082787"} Aug 04 13:13:53.273130 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.272778 2580 scope.go:117] "RemoveContainer" containerID="ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4" Aug 04 13:13:53.282345 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.282327 2580 scope.go:117] "RemoveContainer" containerID="ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4" Aug 04 13:13:53.282600 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:13:53.282582 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4\": container with ID starting with ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4 not found: ID does not exist" containerID="ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4" Aug 04 13:13:53.282675 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.282609 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4"} err="failed to get container status \"ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4\": rpc error: code = NotFound desc = could not find container \"ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4\": container with ID starting with ef74ae4f63f1a58b6e2d3df1250be1e6b28a50ffa4f275e59bfb17b34bb039b4 not found: ID does not exist" Aug 04 13:13:53.297226 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.297201 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6"] Aug 04 13:13:53.305308 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:53.305288 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0c5286-5bd66d7584-gmdv6"] Aug 04 13:13:54.276901 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:54.276871 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81bd2178-dda5-4595-b770-65444440257b" path="/var/lib/kubelet/pods/81bd2178-dda5-4595-b770-65444440257b/volumes" Aug 04 13:13:54.277456 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:54.277434 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="848ab4d4-e393-4bd3-801c-9d1a239f7876" path="/var/lib/kubelet/pods/848ab4d4-e393-4bd3-801c-9d1a239f7876/volumes" Aug 04 13:13:54.515378 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:13:54.515344 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:14:00.772821 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.772782 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk"] Aug 04 13:14:00.773255 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.773238 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="848ab4d4-e393-4bd3-801c-9d1a239f7876" containerName="maas-api" Aug 04 13:14:00.773301 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.773257 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="848ab4d4-e393-4bd3-801c-9d1a239f7876" containerName="maas-api" Aug 04 13:14:00.773301 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.773265 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81bd2178-dda5-4595-b770-65444440257b" containerName="revoke-keys" Aug 04 13:14:00.773301 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.773270 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81bd2178-dda5-4595-b770-65444440257b" containerName="revoke-keys" Aug 04 13:14:00.773405 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.773326 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="848ab4d4-e393-4bd3-801c-9d1a239f7876" containerName="maas-api" Aug 04 13:14:00.773405 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.773335 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81bd2178-dda5-4595-b770-65444440257b" containerName="revoke-keys" Aug 04 13:14:00.776286 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.776266 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" Aug 04 13:14:00.780714 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.780693 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:14:00.790819 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.790794 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk"] Aug 04 13:14:00.892802 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.892768 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5469f0e3-d6cf-4809-9c2a-923543706730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk\" (UID: \"5469f0e3-d6cf-4809-9c2a-923543706730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" Aug 04 13:14:00.994253 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.994202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5469f0e3-d6cf-4809-9c2a-923543706730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk\" (UID: \"5469f0e3-d6cf-4809-9c2a-923543706730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" Aug 04 13:14:00.994796 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:00.994776 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5469f0e3-d6cf-4809-9c2a-923543706730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk\" (UID: \"5469f0e3-d6cf-4809-9c2a-923543706730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" Aug 04 13:14:01.086350 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:01.086314 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" Aug 04 13:14:01.226478 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:01.226438 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk"] Aug 04 13:14:01.229202 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:14:01.229172 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5469f0e3_d6cf_4809_9c2a_923543706730.slice/crio-bf1b085a1318175aa980e77b16815a149f5acd1b7898f6793655123ebe5e93dc WatchSource:0}: Error finding container bf1b085a1318175aa980e77b16815a149f5acd1b7898f6793655123ebe5e93dc: Status 404 returned error can't find the container with id bf1b085a1318175aa980e77b16815a149f5acd1b7898f6793655123ebe5e93dc Aug 04 13:14:01.315519 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:01.315481 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" event={"ID":"5469f0e3-d6cf-4809-9c2a-923543706730","Type":"ContainerStarted","Data":"bf1b085a1318175aa980e77b16815a149f5acd1b7898f6793655123ebe5e93dc"} Aug 04 13:14:02.321725 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:02.321691 2580 generic.go:356] "Generic (PLEG): container finished" podID="5469f0e3-d6cf-4809-9c2a-923543706730" containerID="32da53ca7c3335d9c1776ab8d4040bae613ed7a6881a57f7cfe367c5427e00dc" exitCode=0 Aug 04 13:14:02.322138 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:02.321747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" event={"ID":"5469f0e3-d6cf-4809-9c2a-923543706730","Type":"ContainerDied","Data":"32da53ca7c3335d9c1776ab8d4040bae613ed7a6881a57f7cfe367c5427e00dc"} Aug 04 13:14:03.451472 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:03.451451 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" Aug 04 13:14:03.516512 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:03.516480 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5469f0e3-d6cf-4809-9c2a-923543706730-maas-api-service-ca\") pod \"5469f0e3-d6cf-4809-9c2a-923543706730\" (UID: \"5469f0e3-d6cf-4809-9c2a-923543706730\") " Aug 04 13:14:03.516879 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:03.516856 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5469f0e3-d6cf-4809-9c2a-923543706730-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5469f0e3-d6cf-4809-9c2a-923543706730" (UID: "5469f0e3-d6cf-4809-9c2a-923543706730"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:14:03.617439 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:03.617368 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5469f0e3-d6cf-4809-9c2a-923543706730-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:14:04.333931 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:04.333896 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" Aug 04 13:14:04.334097 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:04.333945 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk" event={"ID":"5469f0e3-d6cf-4809-9c2a-923543706730","Type":"ContainerDied","Data":"bf1b085a1318175aa980e77b16815a149f5acd1b7898f6793655123ebe5e93dc"} Aug 04 13:14:04.334097 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:04.333987 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf1b085a1318175aa980e77b16815a149f5acd1b7898f6793655123ebe5e93dc" Aug 04 13:14:11.178028 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.177996 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv"] Aug 04 13:14:11.178443 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.178207 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" podUID="8d9d36f4-7b1d-497f-97e0-608434cdd168" containerName="maas-api" containerID="cri-o://56b7f954943c6b69ea5c0b9abb359ed1828c108f64df13964203aa3cc9afce89" gracePeriod=30 Aug 04 13:14:11.373876 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.373804 2580 generic.go:356] "Generic (PLEG): container finished" podID="8d9d36f4-7b1d-497f-97e0-608434cdd168" containerID="56b7f954943c6b69ea5c0b9abb359ed1828c108f64df13964203aa3cc9afce89" exitCode=0 Aug 04 13:14:11.374068 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.373898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" event={"ID":"8d9d36f4-7b1d-497f-97e0-608434cdd168","Type":"ContainerDied","Data":"56b7f954943c6b69ea5c0b9abb359ed1828c108f64df13964203aa3cc9afce89"} Aug 04 13:14:11.424544 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.424517 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:14:11.484056 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.483984 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp6kb\" (UniqueName: \"kubernetes.io/projected/8d9d36f4-7b1d-497f-97e0-608434cdd168-kube-api-access-fp6kb\") pod \"8d9d36f4-7b1d-497f-97e0-608434cdd168\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " Aug 04 13:14:11.484056 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.484019 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9d36f4-7b1d-497f-97e0-608434cdd168-openshift-service-ca\") pod \"8d9d36f4-7b1d-497f-97e0-608434cdd168\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " Aug 04 13:14:11.484236 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.484099 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls\") pod \"8d9d36f4-7b1d-497f-97e0-608434cdd168\" (UID: \"8d9d36f4-7b1d-497f-97e0-608434cdd168\") " Aug 04 13:14:11.484479 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.484448 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d9d36f4-7b1d-497f-97e0-608434cdd168-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d9d36f4-7b1d-497f-97e0-608434cdd168" (UID: "8d9d36f4-7b1d-497f-97e0-608434cdd168"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:14:11.486327 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.486299 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d9d36f4-7b1d-497f-97e0-608434cdd168-kube-api-access-fp6kb" (OuterVolumeSpecName: "kube-api-access-fp6kb") pod "8d9d36f4-7b1d-497f-97e0-608434cdd168" (UID: "8d9d36f4-7b1d-497f-97e0-608434cdd168"). InnerVolumeSpecName "kube-api-access-fp6kb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:14:11.486446 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.486362 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d9d36f4-7b1d-497f-97e0-608434cdd168" (UID: "8d9d36f4-7b1d-497f-97e0-608434cdd168"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 13:14:11.585321 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.585290 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d9d36f4-7b1d-497f-97e0-608434cdd168-maas-api-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:14:11.585321 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.585316 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp6kb\" (UniqueName: \"kubernetes.io/projected/8d9d36f4-7b1d-497f-97e0-608434cdd168-kube-api-access-fp6kb\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:14:11.585321 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.585326 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9d36f4-7b1d-497f-97e0-608434cdd168-openshift-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:14:11.706876 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.706817 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk"] Aug 04 13:14:11.708776 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:11.708755 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c1a14d-9gcqk"] Aug 04 13:14:12.276670 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:12.276639 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5469f0e3-d6cf-4809-9c2a-923543706730" path="/var/lib/kubelet/pods/5469f0e3-d6cf-4809-9c2a-923543706730/volumes" Aug 04 13:14:12.385992 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:12.385955 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" Aug 04 13:14:12.385992 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:12.385980 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv" event={"ID":"8d9d36f4-7b1d-497f-97e0-608434cdd168","Type":"ContainerDied","Data":"f2ad62892980187fb313a0b2a3e976b3a5d943f6de2d0b046def85cf3de05519"} Aug 04 13:14:12.386248 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:12.386066 2580 scope.go:117] "RemoveContainer" containerID="56b7f954943c6b69ea5c0b9abb359ed1828c108f64df13964203aa3cc9afce89" Aug 04 13:14:12.406492 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:12.406467 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv"] Aug 04 13:14:12.410994 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:12.410971 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c1a14d-599cd6fccc-7t2wv"] Aug 04 13:14:12.605429 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:12.605395 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:14:14.277246 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:14.277212 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d9d36f4-7b1d-497f-97e0-608434cdd168" path="/var/lib/kubelet/pods/8d9d36f4-7b1d-497f-97e0-608434cdd168/volumes" Aug 04 13:14:20.581863 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:20.581810 2580 scope.go:117] "RemoveContainer" containerID="7c3f15fdff8fb891f0dbd24ddd82551a6674b2be5bbc78d279dbfd4a210dc4a2" Aug 04 13:14:24.873509 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.873474 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk"] Aug 04 13:14:24.874081 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.873975 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d9d36f4-7b1d-497f-97e0-608434cdd168" containerName="maas-api" Aug 04 13:14:24.874081 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.873995 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d9d36f4-7b1d-497f-97e0-608434cdd168" containerName="maas-api" Aug 04 13:14:24.874081 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.874033 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5469f0e3-d6cf-4809-9c2a-923543706730" containerName="revoke-keys" Aug 04 13:14:24.874081 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.874042 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5469f0e3-d6cf-4809-9c2a-923543706730" containerName="revoke-keys" Aug 04 13:14:24.874298 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.874125 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d9d36f4-7b1d-497f-97e0-608434cdd168" containerName="maas-api" Aug 04 13:14:24.874298 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.874139 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5469f0e3-d6cf-4809-9c2a-923543706730" containerName="revoke-keys" Aug 04 13:14:24.878780 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.878760 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:24.882166 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.882139 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-6a00c1-serving-cert\"" Aug 04 13:14:24.882166 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.882157 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 13:14:24.882318 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.882204 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 13:14:24.883348 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.883328 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p76m5\"" Aug 04 13:14:24.891118 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.891097 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk"] Aug 04 13:14:24.996131 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.996090 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8wg7\" (UniqueName: \"kubernetes.io/projected/d342e2df-f388-45ed-9f76-745b8d1a46a7-kube-api-access-s8wg7\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:24.996317 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.996168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:24.996317 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:24.996228 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d342e2df-f388-45ed-9f76-745b8d1a46a7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.097261 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.097218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d342e2df-f388-45ed-9f76-745b8d1a46a7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.097454 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.097277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8wg7\" (UniqueName: \"kubernetes.io/projected/d342e2df-f388-45ed-9f76-745b8d1a46a7-kube-api-access-s8wg7\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.097454 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.097344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.097550 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:14:25.097457 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-serving-cert: secret "maas-api-e2e-shared-a-6a00c1-serving-cert" not found Aug 04 13:14:25.097550 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:14:25.097532 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls podName:d342e2df-f388-45ed-9f76-745b8d1a46a7 nodeName:}" failed. No retries permitted until 2026-08-04 13:14:25.59751265 +0000 UTC m=+2825.833944002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls") pod "maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" (UID: "d342e2df-f388-45ed-9f76-745b8d1a46a7") : secret "maas-api-e2e-shared-a-6a00c1-serving-cert" not found Aug 04 13:14:25.098006 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.097986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d342e2df-f388-45ed-9f76-745b8d1a46a7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.112296 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.112252 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8wg7\" (UniqueName: \"kubernetes.io/projected/d342e2df-f388-45ed-9f76-745b8d1a46a7-kube-api-access-s8wg7\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.600812 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.600777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.603386 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.603364 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.789514 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.789482 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:25.934969 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:25.934944 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk"] Aug 04 13:14:25.936012 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:14:25.935985 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd342e2df_f388_45ed_9f76_745b8d1a46a7.slice/crio-65c93b6a5859d42abb3ac3882709b67c5a3c4469cda3385f2ee01d23d0e79fe5 WatchSource:0}: Error finding container 65c93b6a5859d42abb3ac3882709b67c5a3c4469cda3385f2ee01d23d0e79fe5: Status 404 returned error can't find the container with id 65c93b6a5859d42abb3ac3882709b67c5a3c4469cda3385f2ee01d23d0e79fe5 Aug 04 13:14:26.471096 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:26.471055 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" event={"ID":"d342e2df-f388-45ed-9f76-745b8d1a46a7","Type":"ContainerStarted","Data":"65c93b6a5859d42abb3ac3882709b67c5a3c4469cda3385f2ee01d23d0e79fe5"} Aug 04 13:14:27.478750 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:27.478713 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" event={"ID":"d342e2df-f388-45ed-9f76-745b8d1a46a7","Type":"ContainerStarted","Data":"a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f"} Aug 04 13:14:27.479157 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:27.478810 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:27.500964 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:27.500882 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" podStartSLOduration=2.191562709 podStartE2EDuration="3.500867772s" totalImagesPullingTime="1.309305063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:14:27.500003026 +0000 UTC m=+2827.736434408" watchObservedRunningTime="2026-08-04 13:14:27.500867772 +0000 UTC m=+2827.737299142" Aug 04 13:14:27.607848 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:27.607803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:14:33.490589 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:33.490561 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:14:53.124592 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.124557 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq"] Aug 04 13:14:53.128390 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.128370 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.131161 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.131128 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-c1c995-serving-cert\"" Aug 04 13:14:53.140542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.140520 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq"] Aug 04 13:14:53.225443 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.225407 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnlwf\" (UniqueName: \"kubernetes.io/projected/1b2479db-a867-4267-b681-f504134b34b4-kube-api-access-cnlwf\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.225646 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.225505 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2479db-a867-4267-b681-f504134b34b4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.225646 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.225547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.325936 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.325892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnlwf\" (UniqueName: \"kubernetes.io/projected/1b2479db-a867-4267-b681-f504134b34b4-kube-api-access-cnlwf\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.326138 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.325971 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2479db-a867-4267-b681-f504134b34b4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.326138 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.326007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.326138 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:14:53.326119 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-serving-cert: secret "maas-api-e2e-shared-b-c1c995-serving-cert" not found Aug 04 13:14:53.326317 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:14:53.326182 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls podName:1b2479db-a867-4267-b681-f504134b34b4 nodeName:}" failed. No retries permitted until 2026-08-04 13:14:53.826159892 +0000 UTC m=+2854.062591240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls") pod "maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" (UID: "1b2479db-a867-4267-b681-f504134b34b4") : secret "maas-api-e2e-shared-b-c1c995-serving-cert" not found Aug 04 13:14:53.326621 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.326599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2479db-a867-4267-b681-f504134b34b4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.336372 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.336339 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnlwf\" (UniqueName: \"kubernetes.io/projected/1b2479db-a867-4267-b681-f504134b34b4-kube-api-access-cnlwf\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.831687 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.831654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:53.834214 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:53.834189 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls\") pod \"maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:54.039426 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:54.039398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:54.180537 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:54.180508 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq"] Aug 04 13:14:54.182243 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:14:54.182210 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b2479db_a867_4267_b681_f504134b34b4.slice/crio-097cd6866cea34818d08af16443bb0a66b88892d2ddcd8a160f06b0e69d4cdd0 WatchSource:0}: Error finding container 097cd6866cea34818d08af16443bb0a66b88892d2ddcd8a160f06b0e69d4cdd0: Status 404 returned error can't find the container with id 097cd6866cea34818d08af16443bb0a66b88892d2ddcd8a160f06b0e69d4cdd0 Aug 04 13:14:54.623572 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:54.623534 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" event={"ID":"1b2479db-a867-4267-b681-f504134b34b4","Type":"ContainerStarted","Data":"097cd6866cea34818d08af16443bb0a66b88892d2ddcd8a160f06b0e69d4cdd0"} Aug 04 13:14:55.631579 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:55.631538 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" event={"ID":"1b2479db-a867-4267-b681-f504134b34b4","Type":"ContainerStarted","Data":"a5f124ef75f975d3b7c7802efc94e1109f421b5276a2895362a71983f137061f"} Aug 04 13:14:55.632026 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:55.631711 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:14:55.649587 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:55.649542 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" podStartSLOduration=2.0787991 podStartE2EDuration="2.649523326s" totalImagesPullingTime="570.724226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:14:55.64918987 +0000 UTC m=+2855.885621241" watchObservedRunningTime="2026-08-04 13:14:55.649523326 +0000 UTC m=+2855.885954693" Aug 04 13:14:56.591136 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:14:56.591098 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:15:00.138334 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.138293 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx"] Aug 04 13:15:00.142602 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.142577 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m"] Aug 04 13:15:00.142752 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.142694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx" Aug 04 13:15:00.145816 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.145795 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw"] Aug 04 13:15:00.145972 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.145928 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" Aug 04 13:15:00.146037 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.145932 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:15:00.149036 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.149014 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw" Aug 04 13:15:00.151347 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.151287 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx"] Aug 04 13:15:00.157918 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.157890 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx" Aug 04 13:15:00.159723 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.159614 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw"] Aug 04 13:15:00.167792 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.165535 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m"] Aug 04 13:15:00.172566 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.172543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" Aug 04 13:15:00.209374 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.209352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw" Aug 04 13:15:00.363149 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.363119 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m"] Aug 04 13:15:00.366352 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:15:00.366321 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20fe25bd_f2dc_4af9_aac0_dfefe64cce1f.slice/crio-38d22d7e0cb8652d4f6d5345f5136ef601bee90a1cbfa9ec330060cda1164b81 WatchSource:0}: Error finding container 38d22d7e0cb8652d4f6d5345f5136ef601bee90a1cbfa9ec330060cda1164b81: Status 404 returned error can't find the container with id 38d22d7e0cb8652d4f6d5345f5136ef601bee90a1cbfa9ec330060cda1164b81 Aug 04 13:15:00.393635 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.393612 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw"] Aug 04 13:15:00.394941 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:15:00.394911 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54bcac1f_e1d9_4ed4_a6e6_027b6b2c0764.slice/crio-06759acf372c26006305f4763741dbebe05767b03421fbecad8807376ce34ac0 WatchSource:0}: Error finding container 06759acf372c26006305f4763741dbebe05767b03421fbecad8807376ce34ac0: Status 404 returned error can't find the container with id 06759acf372c26006305f4763741dbebe05767b03421fbecad8807376ce34ac0 Aug 04 13:15:00.539476 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.539453 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx"] Aug 04 13:15:00.541095 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:15:00.541071 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ee6716_8e41_4074_b4e2_d0322d9b75c3.slice/crio-99cf9ac38818e2af90639ba25ac42d5d1959ac18b22546744d160b0fbdcc003b WatchSource:0}: Error finding container 99cf9ac38818e2af90639ba25ac42d5d1959ac18b22546744d160b0fbdcc003b: Status 404 returned error can't find the container with id 99cf9ac38818e2af90639ba25ac42d5d1959ac18b22546744d160b0fbdcc003b Aug 04 13:15:00.660137 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.660035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw" event={"ID":"54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764","Type":"ContainerStarted","Data":"06759acf372c26006305f4763741dbebe05767b03421fbecad8807376ce34ac0"} Aug 04 13:15:00.661627 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.661600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" event={"ID":"20fe25bd-f2dc-4af9-aac0-dfefe64cce1f","Type":"ContainerStarted","Data":"87afd4dbc1ff16d6fb6548803f569141d0aac4128a88292a63817c6a8cd04537"} Aug 04 13:15:00.661752 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.661650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" event={"ID":"20fe25bd-f2dc-4af9-aac0-dfefe64cce1f","Type":"ContainerStarted","Data":"38d22d7e0cb8652d4f6d5345f5136ef601bee90a1cbfa9ec330060cda1164b81"} Aug 04 13:15:00.662820 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.662797 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx" event={"ID":"78ee6716-8e41-4074-b4e2-d0322d9b75c3","Type":"ContainerStarted","Data":"99cf9ac38818e2af90639ba25ac42d5d1959ac18b22546744d160b0fbdcc003b"} Aug 04 13:15:00.680050 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:00.680002 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" podStartSLOduration=0.679987647 podStartE2EDuration="679.987647ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:15:00.67773967 +0000 UTC m=+2860.914171041" watchObservedRunningTime="2026-08-04 13:15:00.679987647 +0000 UTC m=+2860.916419016" Aug 04 13:15:01.641791 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:01.641720 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:15:01.670446 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:01.670415 2580 generic.go:356] "Generic (PLEG): container finished" podID="78ee6716-8e41-4074-b4e2-d0322d9b75c3" containerID="f48cae314983caa6ff355009e2802769d177d229ba77d963f07a630859e53f2f" exitCode=0 Aug 04 13:15:01.670648 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:01.670507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx" event={"ID":"78ee6716-8e41-4074-b4e2-d0322d9b75c3","Type":"ContainerDied","Data":"f48cae314983caa6ff355009e2802769d177d229ba77d963f07a630859e53f2f"} Aug 04 13:15:01.672010 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:01.671987 2580 generic.go:356] "Generic (PLEG): container finished" podID="54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764" containerID="e02f90dfbfcca524a4dfd489998f56fdd84183441175d5ff420b833f2a72318a" exitCode=0 Aug 04 13:15:01.672162 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:01.672045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw" event={"ID":"54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764","Type":"ContainerDied","Data":"e02f90dfbfcca524a4dfd489998f56fdd84183441175d5ff420b833f2a72318a"} Aug 04 13:15:01.674569 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:01.674545 2580 generic.go:356] "Generic (PLEG): container finished" podID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" containerID="87afd4dbc1ff16d6fb6548803f569141d0aac4128a88292a63817c6a8cd04537" exitCode=7 Aug 04 13:15:01.674665 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:01.674609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" event={"ID":"20fe25bd-f2dc-4af9-aac0-dfefe64cce1f","Type":"ContainerDied","Data":"87afd4dbc1ff16d6fb6548803f569141d0aac4128a88292a63817c6a8cd04537"} Aug 04 13:15:01.674870 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:01.674855 2580 scope.go:117] "RemoveContainer" containerID="87afd4dbc1ff16d6fb6548803f569141d0aac4128a88292a63817c6a8cd04537" Aug 04 13:15:02.681447 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:02.681418 2580 generic.go:356] "Generic (PLEG): container finished" podID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" containerID="8c37f7af96374181498f3b60cd358924e23f37c613103a029ba016e0a40b91ad" exitCode=0 Aug 04 13:15:02.681929 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:02.681489 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" event={"ID":"20fe25bd-f2dc-4af9-aac0-dfefe64cce1f","Type":"ContainerDied","Data":"8c37f7af96374181498f3b60cd358924e23f37c613103a029ba016e0a40b91ad"} Aug 04 13:15:02.681929 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:02.681533 2580 scope.go:117] "RemoveContainer" containerID="87afd4dbc1ff16d6fb6548803f569141d0aac4128a88292a63817c6a8cd04537" Aug 04 13:15:02.812548 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:02.812521 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw" Aug 04 13:15:02.853678 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:02.853654 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx" Aug 04 13:15:03.688322 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:03.688293 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx" Aug 04 13:15:03.688756 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:03.688292 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764155-tb5xx" event={"ID":"78ee6716-8e41-4074-b4e2-d0322d9b75c3","Type":"ContainerDied","Data":"99cf9ac38818e2af90639ba25ac42d5d1959ac18b22546744d160b0fbdcc003b"} Aug 04 13:15:03.688756 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:03.688431 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99cf9ac38818e2af90639ba25ac42d5d1959ac18b22546744d160b0fbdcc003b" Aug 04 13:15:03.690720 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:03.690131 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw" event={"ID":"54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764","Type":"ContainerDied","Data":"06759acf372c26006305f4763741dbebe05767b03421fbecad8807376ce34ac0"} Aug 04 13:15:03.690720 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:03.690178 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06759acf372c26006305f4763741dbebe05767b03421fbecad8807376ce34ac0" Aug 04 13:15:03.690720 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:03.690229 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw" Aug 04 13:15:03.822237 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:03.822208 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" Aug 04 13:15:04.702325 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:04.702294 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" Aug 04 13:15:04.702325 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:04.702322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m" event={"ID":"20fe25bd-f2dc-4af9-aac0-dfefe64cce1f","Type":"ContainerDied","Data":"38d22d7e0cb8652d4f6d5345f5136ef601bee90a1cbfa9ec330060cda1164b81"} Aug 04 13:15:04.702824 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:04.702363 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38d22d7e0cb8652d4f6d5345f5136ef601bee90a1cbfa9ec330060cda1164b81" Aug 04 13:15:28.209779 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:15:28.209686 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:16:01.000152 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:16:01.000118 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:16:22.504320 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:16:22.504284 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:16:37.830117 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:16:37.830075 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:16:49.316129 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:16:49.316048 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:17:04.420161 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:17:04.420128 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:17:16.214704 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:17:16.214668 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:17:20.502586 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:17:20.502555 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:17:20.511592 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:17:20.511568 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:17:31.102208 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:17:31.102174 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:17:42.312985 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:17:42.312948 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:17:57.414037 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:17:57.414002 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:18:08.706196 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:18:08.706162 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:18:20.635095 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:18:20.635061 2580 scope.go:117] "RemoveContainer" containerID="7cacf5ca0760acd6f85691429847dd33145f55d09d8624f82a8dd158996b9df8" Aug 04 13:18:23.920348 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:18:23.920311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:18:35.697464 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:18:35.697425 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:18:50.595540 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:18:50.595494 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:19:00.413714 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:19:00.413680 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:19:17.713322 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:19:17.713285 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:19:20.655489 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:19:20.655453 2580 scope.go:117] "RemoveContainer" containerID="c19ac9df223e91031b5c3303927fe684bd638ea6f6aea93c253f29e25025a6c3" Aug 04 13:19:44.399961 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:19:44.399880 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:20:20.676269 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:20:20.676232 2580 scope.go:117] "RemoveContainer" containerID="32da53ca7c3335d9c1776ab8d4040bae613ed7a6881a57f7cfe367c5427e00dc" Aug 04 13:20:20.687218 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:20:20.687148 2580 scope.go:117] "RemoveContainer" containerID="58d33286a21c7ccbf56bf97f9e85b5ace5882df848cc53111315ebfacba3904f" Aug 04 13:20:23.019674 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:20:23.019641 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:20:34.645719 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:20:34.645686 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:21:01.913808 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:21:01.913764 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:21:12.798184 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:21:12.798098 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:21:32.118044 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:21:32.118003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:22:07.500805 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:22:07.500765 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:22:20.536195 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:22:20.536169 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:22:20.546617 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:22:20.546589 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:22:45.515696 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:22:45.515608 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:22:54.808250 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:22:54.808196 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:23:10.002607 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:10.002565 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:23:17.549470 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549433 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-140604917820976"] Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549774 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ee6716-8e41-4074-b4e2-d0322d9b75c3" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549784 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ee6716-8e41-4074-b4e2-d0322d9b75c3" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549793 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549799 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549808 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549814 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549825 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549846 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549903 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549911 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549917 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" containerName="cleanup" Aug 04 13:23:17.549974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.549924 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ee6716-8e41-4074-b4e2-d0322d9b75c3" containerName="cleanup" Aug 04 13:23:17.553119 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.553102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917820976" Aug 04 13:23:17.556421 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.556398 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4l88d\"" Aug 04 13:23:17.565428 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.565401 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917820976"] Aug 04 13:23:17.727549 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.727511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cn9k\" (UniqueName: \"kubernetes.io/projected/dea20005-9620-45f3-ab92-e5e80c028d04-kube-api-access-4cn9k\") pod \"test-curl-7696-140604917820976\" (UID: \"dea20005-9620-45f3-ab92-e5e80c028d04\") " pod="opendatahub/test-curl-7696-140604917820976" Aug 04 13:23:17.828367 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.828327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cn9k\" (UniqueName: \"kubernetes.io/projected/dea20005-9620-45f3-ab92-e5e80c028d04-kube-api-access-4cn9k\") pod \"test-curl-7696-140604917820976\" (UID: \"dea20005-9620-45f3-ab92-e5e80c028d04\") " pod="opendatahub/test-curl-7696-140604917820976" Aug 04 13:23:17.839069 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.839047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cn9k\" (UniqueName: \"kubernetes.io/projected/dea20005-9620-45f3-ab92-e5e80c028d04-kube-api-access-4cn9k\") pod \"test-curl-7696-140604917820976\" (UID: \"dea20005-9620-45f3-ab92-e5e80c028d04\") " pod="opendatahub/test-curl-7696-140604917820976" Aug 04 13:23:17.861951 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.861908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917820976" Aug 04 13:23:17.998845 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:17.998802 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917820976"] Aug 04 13:23:18.000291 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:23:18.000264 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddea20005_9620_45f3_ab92_e5e80c028d04.slice/crio-de834ded51e64e9468221a344a71a423465a84cc92f05b161ad091886af5a9f0 WatchSource:0}: Error finding container de834ded51e64e9468221a344a71a423465a84cc92f05b161ad091886af5a9f0: Status 404 returned error can't find the container with id de834ded51e64e9468221a344a71a423465a84cc92f05b161ad091886af5a9f0 Aug 04 13:23:18.001720 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:18.001698 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 13:23:18.331639 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:18.331597 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917820976" event={"ID":"dea20005-9620-45f3-ab92-e5e80c028d04","Type":"ContainerStarted","Data":"de834ded51e64e9468221a344a71a423465a84cc92f05b161ad091886af5a9f0"} Aug 04 13:23:19.100631 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:19.100593 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:23:19.339389 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:19.339353 2580 generic.go:356] "Generic (PLEG): container finished" podID="dea20005-9620-45f3-ab92-e5e80c028d04" containerID="798beec66dbca2f5df793f06673209f71a9d91ffccbad9f1e797e3f8d1a8c7de" exitCode=0 Aug 04 13:23:19.339581 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:19.339447 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917820976" event={"ID":"dea20005-9620-45f3-ab92-e5e80c028d04","Type":"ContainerDied","Data":"798beec66dbca2f5df793f06673209f71a9d91ffccbad9f1e797e3f8d1a8c7de"} Aug 04 13:23:20.482165 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.482139 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917820976" Aug 04 13:23:20.530541 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.530509 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7696-140604917820976_dea20005-9620-45f3-ab92-e5e80c028d04/test-curl-7696-140604917820976/0.log" Aug 04 13:23:20.558153 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.558114 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-140604917820976"] Aug 04 13:23:20.562652 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.562621 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-140604917820976"] Aug 04 13:23:20.652680 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.652519 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cn9k\" (UniqueName: \"kubernetes.io/projected/dea20005-9620-45f3-ab92-e5e80c028d04-kube-api-access-4cn9k\") pod \"dea20005-9620-45f3-ab92-e5e80c028d04\" (UID: \"dea20005-9620-45f3-ab92-e5e80c028d04\") " Aug 04 13:23:20.666388 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.654878 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dea20005-9620-45f3-ab92-e5e80c028d04-kube-api-access-4cn9k" (OuterVolumeSpecName: "kube-api-access-4cn9k") pod "dea20005-9620-45f3-ab92-e5e80c028d04" (UID: "dea20005-9620-45f3-ab92-e5e80c028d04"). InnerVolumeSpecName "kube-api-access-4cn9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:23:20.681589 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.681552 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-140604917892144"] Aug 04 13:23:20.681917 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.681902 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dea20005-9620-45f3-ab92-e5e80c028d04" containerName="test-curl-7696-140604917820976" Aug 04 13:23:20.681968 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.681919 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dea20005-9620-45f3-ab92-e5e80c028d04" containerName="test-curl-7696-140604917820976" Aug 04 13:23:20.682018 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.681991 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dea20005-9620-45f3-ab92-e5e80c028d04" containerName="test-curl-7696-140604917820976" Aug 04 13:23:20.684801 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.684782 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917892144" Aug 04 13:23:20.693457 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.693426 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917892144"] Aug 04 13:23:20.753145 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.753105 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cn9k\" (UniqueName: \"kubernetes.io/projected/dea20005-9620-45f3-ab92-e5e80c028d04-kube-api-access-4cn9k\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:23:20.853914 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.853867 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6v6v\" (UniqueName: \"kubernetes.io/projected/98a1583f-6eb5-4fe7-adb7-ce96f9f14746-kube-api-access-k6v6v\") pod \"test-curl-7696-140604917892144\" (UID: \"98a1583f-6eb5-4fe7-adb7-ce96f9f14746\") " pod="opendatahub/test-curl-7696-140604917892144" Aug 04 13:23:20.955035 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.954937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6v6v\" (UniqueName: \"kubernetes.io/projected/98a1583f-6eb5-4fe7-adb7-ce96f9f14746-kube-api-access-k6v6v\") pod \"test-curl-7696-140604917892144\" (UID: \"98a1583f-6eb5-4fe7-adb7-ce96f9f14746\") " pod="opendatahub/test-curl-7696-140604917892144" Aug 04 13:23:20.964591 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.964561 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6v6v\" (UniqueName: \"kubernetes.io/projected/98a1583f-6eb5-4fe7-adb7-ce96f9f14746-kube-api-access-k6v6v\") pod \"test-curl-7696-140604917892144\" (UID: \"98a1583f-6eb5-4fe7-adb7-ce96f9f14746\") " pod="opendatahub/test-curl-7696-140604917892144" Aug 04 13:23:20.995184 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:20.995146 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917892144" Aug 04 13:23:21.137149 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:21.137121 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917892144"] Aug 04 13:23:21.138563 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:23:21.138535 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98a1583f_6eb5_4fe7_adb7_ce96f9f14746.slice/crio-cccc31039edcc9f2745c806dc7ae6592eda676b1aa3d47d1fda8fb2fdddcc284 WatchSource:0}: Error finding container cccc31039edcc9f2745c806dc7ae6592eda676b1aa3d47d1fda8fb2fdddcc284: Status 404 returned error can't find the container with id cccc31039edcc9f2745c806dc7ae6592eda676b1aa3d47d1fda8fb2fdddcc284 Aug 04 13:23:21.350964 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:21.350927 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917820976" Aug 04 13:23:21.350964 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:21.350964 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de834ded51e64e9468221a344a71a423465a84cc92f05b161ad091886af5a9f0" Aug 04 13:23:21.352160 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:21.352131 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917892144" event={"ID":"98a1583f-6eb5-4fe7-adb7-ce96f9f14746","Type":"ContainerStarted","Data":"cccc31039edcc9f2745c806dc7ae6592eda676b1aa3d47d1fda8fb2fdddcc284"} Aug 04 13:23:21.363449 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:23:21.363420 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-140604917820976\" is forbidden: User \"system:node:ip-10-0-132-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-96.ec2.internal' and this object" podUID="dea20005-9620-45f3-ab92-e5e80c028d04" pod="opendatahub/test-curl-7696-140604917820976" Aug 04 13:23:22.276988 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:22.276954 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dea20005-9620-45f3-ab92-e5e80c028d04" path="/var/lib/kubelet/pods/dea20005-9620-45f3-ab92-e5e80c028d04/volumes" Aug 04 13:23:22.359928 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:22.359891 2580 generic.go:356] "Generic (PLEG): container finished" podID="98a1583f-6eb5-4fe7-adb7-ce96f9f14746" containerID="1738003cff37ba10e1c4944649320d2b1499fe0a6602cdff1fb63f14eab72090" exitCode=0 Aug 04 13:23:22.360096 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:22.359972 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917892144" event={"ID":"98a1583f-6eb5-4fe7-adb7-ce96f9f14746","Type":"ContainerDied","Data":"1738003cff37ba10e1c4944649320d2b1499fe0a6602cdff1fb63f14eab72090"} Aug 04 13:23:23.505544 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.505515 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917892144" Aug 04 13:23:23.527682 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.527654 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7696-140604917892144_98a1583f-6eb5-4fe7-adb7-ce96f9f14746/test-curl-7696-140604917892144/0.log" Aug 04 13:23:23.552584 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.552550 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-140604917892144"] Aug 04 13:23:23.559469 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.559444 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-140604917892144"] Aug 04 13:23:23.577419 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.577390 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6v6v\" (UniqueName: \"kubernetes.io/projected/98a1583f-6eb5-4fe7-adb7-ce96f9f14746-kube-api-access-k6v6v\") pod \"98a1583f-6eb5-4fe7-adb7-ce96f9f14746\" (UID: \"98a1583f-6eb5-4fe7-adb7-ce96f9f14746\") " Aug 04 13:23:23.579604 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.579576 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a1583f-6eb5-4fe7-adb7-ce96f9f14746-kube-api-access-k6v6v" (OuterVolumeSpecName: "kube-api-access-k6v6v") pod "98a1583f-6eb5-4fe7-adb7-ce96f9f14746" (UID: "98a1583f-6eb5-4fe7-adb7-ce96f9f14746"). InnerVolumeSpecName "kube-api-access-k6v6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:23:23.678381 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.678291 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6v6v\" (UniqueName: \"kubernetes.io/projected/98a1583f-6eb5-4fe7-adb7-ce96f9f14746-kube-api-access-k6v6v\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:23:23.737277 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.737238 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:23.737620 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.737604 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a1583f-6eb5-4fe7-adb7-ce96f9f14746" containerName="test-curl-7696-140604917892144" Aug 04 13:23:23.737668 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.737623 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a1583f-6eb5-4fe7-adb7-ce96f9f14746" containerName="test-curl-7696-140604917892144" Aug 04 13:23:23.737708 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.737699 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a1583f-6eb5-4fe7-adb7-ce96f9f14746" containerName="test-curl-7696-140604917892144" Aug 04 13:23:23.740655 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.740639 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:23.745803 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.745775 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:23.779203 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.779167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dklr\" (UniqueName: \"kubernetes.io/projected/b6681959-0391-4991-b626-b876fb70b24f-kube-api-access-2dklr\") pod \"test-curl-7696-140604917706016\" (UID: \"b6681959-0391-4991-b626-b876fb70b24f\") " pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:23.880547 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.880512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dklr\" (UniqueName: \"kubernetes.io/projected/b6681959-0391-4991-b626-b876fb70b24f-kube-api-access-2dklr\") pod \"test-curl-7696-140604917706016\" (UID: \"b6681959-0391-4991-b626-b876fb70b24f\") " pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:23.891988 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:23.891959 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dklr\" (UniqueName: \"kubernetes.io/projected/b6681959-0391-4991-b626-b876fb70b24f-kube-api-access-2dklr\") pod \"test-curl-7696-140604917706016\" (UID: \"b6681959-0391-4991-b626-b876fb70b24f\") " pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:24.051786 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:24.051710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:24.189243 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:24.189216 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:24.190656 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:23:24.190626 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6681959_0391_4991_b626_b876fb70b24f.slice/crio-e0a31fac37a589ebfdcf72177c69e4ec736bf8204a9c5d95608356c53080e73c WatchSource:0}: Error finding container e0a31fac37a589ebfdcf72177c69e4ec736bf8204a9c5d95608356c53080e73c: Status 404 returned error can't find the container with id e0a31fac37a589ebfdcf72177c69e4ec736bf8204a9c5d95608356c53080e73c Aug 04 13:23:24.276882 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:24.276818 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98a1583f-6eb5-4fe7-adb7-ce96f9f14746" path="/var/lib/kubelet/pods/98a1583f-6eb5-4fe7-adb7-ce96f9f14746/volumes" Aug 04 13:23:24.371499 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:24.371475 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917892144" Aug 04 13:23:24.371691 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:24.371517 2580 scope.go:117] "RemoveContainer" containerID="1738003cff37ba10e1c4944649320d2b1499fe0a6602cdff1fb63f14eab72090" Aug 04 13:23:24.372880 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:24.372858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917706016" event={"ID":"b6681959-0391-4991-b626-b876fb70b24f","Type":"ContainerStarted","Data":"e0a31fac37a589ebfdcf72177c69e4ec736bf8204a9c5d95608356c53080e73c"} Aug 04 13:23:25.380652 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:25.380616 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6681959-0391-4991-b626-b876fb70b24f" containerID="2d50105277d61514b53bc9b32c5740a1df2e0bf3b20c47aa134b9f69215e0123" exitCode=0 Aug 04 13:23:25.381128 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:25.380678 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917706016" event={"ID":"b6681959-0391-4991-b626-b876fb70b24f","Type":"ContainerDied","Data":"2d50105277d61514b53bc9b32c5740a1df2e0bf3b20c47aa134b9f69215e0123"} Aug 04 13:23:26.520740 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.520713 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:26.543500 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.543470 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7696-140604917706016_b6681959-0391-4991-b626-b876fb70b24f/test-curl-7696-140604917706016/0.log" Aug 04 13:23:26.567097 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.567065 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:26.575122 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.575096 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:26.600857 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.600815 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dklr\" (UniqueName: \"kubernetes.io/projected/b6681959-0391-4991-b626-b876fb70b24f-kube-api-access-2dklr\") pod \"b6681959-0391-4991-b626-b876fb70b24f\" (UID: \"b6681959-0391-4991-b626-b876fb70b24f\") " Aug 04 13:23:26.602977 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.602952 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6681959-0391-4991-b626-b876fb70b24f-kube-api-access-2dklr" (OuterVolumeSpecName: "kube-api-access-2dklr") pod "b6681959-0391-4991-b626-b876fb70b24f" (UID: "b6681959-0391-4991-b626-b876fb70b24f"). InnerVolumeSpecName "kube-api-access-2dklr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:23:26.698999 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.698915 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:26.699284 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.699272 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6681959-0391-4991-b626-b876fb70b24f" containerName="test-curl-7696-140604917706016" Aug 04 13:23:26.699326 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.699286 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6681959-0391-4991-b626-b876fb70b24f" containerName="test-curl-7696-140604917706016" Aug 04 13:23:26.699358 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.699337 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6681959-0391-4991-b626-b876fb70b24f" containerName="test-curl-7696-140604917706016" Aug 04 13:23:26.701497 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.701471 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dklr\" (UniqueName: \"kubernetes.io/projected/b6681959-0391-4991-b626-b876fb70b24f-kube-api-access-2dklr\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:23:26.702320 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.702305 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:26.712732 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.712701 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:26.802002 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.801962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqcfh\" (UniqueName: \"kubernetes.io/projected/29ed7751-9aef-4b0d-b41d-eb6e41fd50e5-kube-api-access-qqcfh\") pod \"test-curl-7696-140604917706016\" (UID: \"29ed7751-9aef-4b0d-b41d-eb6e41fd50e5\") " pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:26.902447 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.902407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqcfh\" (UniqueName: \"kubernetes.io/projected/29ed7751-9aef-4b0d-b41d-eb6e41fd50e5-kube-api-access-qqcfh\") pod \"test-curl-7696-140604917706016\" (UID: \"29ed7751-9aef-4b0d-b41d-eb6e41fd50e5\") " pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:26.914475 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:26.914447 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqcfh\" (UniqueName: \"kubernetes.io/projected/29ed7751-9aef-4b0d-b41d-eb6e41fd50e5-kube-api-access-qqcfh\") pod \"test-curl-7696-140604917706016\" (UID: \"29ed7751-9aef-4b0d-b41d-eb6e41fd50e5\") " pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:27.012163 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:27.012073 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:27.152945 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:27.152920 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:27.154081 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:23:27.154056 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29ed7751_9aef_4b0d_b41d_eb6e41fd50e5.slice/crio-ce253ab701d823a74657ca52e25d036e23d5c47341fecdd7aa81634082df8117 WatchSource:0}: Error finding container ce253ab701d823a74657ca52e25d036e23d5c47341fecdd7aa81634082df8117: Status 404 returned error can't find the container with id ce253ab701d823a74657ca52e25d036e23d5c47341fecdd7aa81634082df8117 Aug 04 13:23:27.392095 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:27.392051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917706016" event={"ID":"29ed7751-9aef-4b0d-b41d-eb6e41fd50e5","Type":"ContainerStarted","Data":"ce253ab701d823a74657ca52e25d036e23d5c47341fecdd7aa81634082df8117"} Aug 04 13:23:27.393334 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:27.393314 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0a31fac37a589ebfdcf72177c69e4ec736bf8204a9c5d95608356c53080e73c" Aug 04 13:23:27.393428 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:27.393335 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:27.405578 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:27.405543 2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7696-140604917706016" oldPodUID="b6681959-0391-4991-b626-b876fb70b24f" podUID="29ed7751-9aef-4b0d-b41d-eb6e41fd50e5" Aug 04 13:23:28.277541 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:28.277505 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6681959-0391-4991-b626-b876fb70b24f" path="/var/lib/kubelet/pods/b6681959-0391-4991-b626-b876fb70b24f/volumes" Aug 04 13:23:28.400295 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:28.400258 2580 generic.go:356] "Generic (PLEG): container finished" podID="29ed7751-9aef-4b0d-b41d-eb6e41fd50e5" containerID="ea73ea247a17b6855710f9d92a6b0bb7f9c0601f8c2b7d2b84f86395f6f3e439" exitCode=0 Aug 04 13:23:28.400487 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:28.400348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917706016" event={"ID":"29ed7751-9aef-4b0d-b41d-eb6e41fd50e5","Type":"ContainerDied","Data":"ea73ea247a17b6855710f9d92a6b0bb7f9c0601f8c2b7d2b84f86395f6f3e439"} Aug 04 13:23:29.541300 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.541275 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:29.563920 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.563888 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7696-140604917706016_29ed7751-9aef-4b0d-b41d-eb6e41fd50e5/test-curl-7696-140604917706016/0.log" Aug 04 13:23:29.589552 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.589517 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:29.593408 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.593382 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-140604917706016"] Aug 04 13:23:29.625672 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.625640 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqcfh\" (UniqueName: \"kubernetes.io/projected/29ed7751-9aef-4b0d-b41d-eb6e41fd50e5-kube-api-access-qqcfh\") pod \"29ed7751-9aef-4b0d-b41d-eb6e41fd50e5\" (UID: \"29ed7751-9aef-4b0d-b41d-eb6e41fd50e5\") " Aug 04 13:23:29.627878 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.627855 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ed7751-9aef-4b0d-b41d-eb6e41fd50e5-kube-api-access-qqcfh" (OuterVolumeSpecName: "kube-api-access-qqcfh") pod "29ed7751-9aef-4b0d-b41d-eb6e41fd50e5" (UID: "29ed7751-9aef-4b0d-b41d-eb6e41fd50e5"). InnerVolumeSpecName "kube-api-access-qqcfh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:23:29.720496 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.720411 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-140604917805392"] Aug 04 13:23:29.720765 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.720751 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ed7751-9aef-4b0d-b41d-eb6e41fd50e5" containerName="test-curl-7696-140604917706016" Aug 04 13:23:29.720809 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.720767 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ed7751-9aef-4b0d-b41d-eb6e41fd50e5" containerName="test-curl-7696-140604917706016" Aug 04 13:23:29.720869 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.720855 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ed7751-9aef-4b0d-b41d-eb6e41fd50e5" containerName="test-curl-7696-140604917706016" Aug 04 13:23:29.723884 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.723867 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917805392" Aug 04 13:23:29.726582 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.726561 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqcfh\" (UniqueName: \"kubernetes.io/projected/29ed7751-9aef-4b0d-b41d-eb6e41fd50e5-kube-api-access-qqcfh\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:23:29.731906 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.731474 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917805392"] Aug 04 13:23:29.827188 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.827138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98fk2\" (UniqueName: \"kubernetes.io/projected/f450e974-ffa7-479f-983b-71eaab8a1e3a-kube-api-access-98fk2\") pod \"test-curl-7696-140604917805392\" (UID: \"f450e974-ffa7-479f-983b-71eaab8a1e3a\") " pod="opendatahub/test-curl-7696-140604917805392" Aug 04 13:23:29.928155 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.928117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98fk2\" (UniqueName: \"kubernetes.io/projected/f450e974-ffa7-479f-983b-71eaab8a1e3a-kube-api-access-98fk2\") pod \"test-curl-7696-140604917805392\" (UID: \"f450e974-ffa7-479f-983b-71eaab8a1e3a\") " pod="opendatahub/test-curl-7696-140604917805392" Aug 04 13:23:29.938167 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:29.938134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98fk2\" (UniqueName: \"kubernetes.io/projected/f450e974-ffa7-479f-983b-71eaab8a1e3a-kube-api-access-98fk2\") pod \"test-curl-7696-140604917805392\" (UID: \"f450e974-ffa7-479f-983b-71eaab8a1e3a\") " pod="opendatahub/test-curl-7696-140604917805392" Aug 04 13:23:30.035443 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:30.035351 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917805392" Aug 04 13:23:30.171611 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:30.171585 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-140604917805392"] Aug 04 13:23:30.173767 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:23:30.173737 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf450e974_ffa7_479f_983b_71eaab8a1e3a.slice/crio-1fd3a2d7c9d405d40161c079caf575c5b36c72c3b361a44dd9acbb4832f755bc WatchSource:0}: Error finding container 1fd3a2d7c9d405d40161c079caf575c5b36c72c3b361a44dd9acbb4832f755bc: Status 404 returned error can't find the container with id 1fd3a2d7c9d405d40161c079caf575c5b36c72c3b361a44dd9acbb4832f755bc Aug 04 13:23:30.277620 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:30.277590 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29ed7751-9aef-4b0d-b41d-eb6e41fd50e5" path="/var/lib/kubelet/pods/29ed7751-9aef-4b0d-b41d-eb6e41fd50e5/volumes" Aug 04 13:23:30.412796 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:30.412767 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917706016" Aug 04 13:23:30.413031 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:30.412799 2580 scope.go:117] "RemoveContainer" containerID="ea73ea247a17b6855710f9d92a6b0bb7f9c0601f8c2b7d2b84f86395f6f3e439" Aug 04 13:23:30.414220 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:30.414136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917805392" event={"ID":"f450e974-ffa7-479f-983b-71eaab8a1e3a","Type":"ContainerStarted","Data":"1fd3a2d7c9d405d40161c079caf575c5b36c72c3b361a44dd9acbb4832f755bc"} Aug 04 13:23:31.422240 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:31.422202 2580 generic.go:356] "Generic (PLEG): container finished" podID="f450e974-ffa7-479f-983b-71eaab8a1e3a" containerID="3f108d41119354b74fbdc523262358f2fe1ce17ca885d39d9c41aae8272cef2c" exitCode=0 Aug 04 13:23:31.422628 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:31.422266 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-140604917805392" event={"ID":"f450e974-ffa7-479f-983b-71eaab8a1e3a","Type":"ContainerDied","Data":"3f108d41119354b74fbdc523262358f2fe1ce17ca885d39d9c41aae8272cef2c"} Aug 04 13:23:32.572432 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:32.572399 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917805392" Aug 04 13:23:32.596590 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:32.596554 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7696-140604917805392_f450e974-ffa7-479f-983b-71eaab8a1e3a/test-curl-7696-140604917805392/0.log" Aug 04 13:23:32.623368 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:32.623327 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-140604917805392"] Aug 04 13:23:32.627962 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:32.627933 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-140604917805392"] Aug 04 13:23:32.654109 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:32.654078 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98fk2\" (UniqueName: \"kubernetes.io/projected/f450e974-ffa7-479f-983b-71eaab8a1e3a-kube-api-access-98fk2\") pod \"f450e974-ffa7-479f-983b-71eaab8a1e3a\" (UID: \"f450e974-ffa7-479f-983b-71eaab8a1e3a\") " Aug 04 13:23:32.656329 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:32.656306 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f450e974-ffa7-479f-983b-71eaab8a1e3a-kube-api-access-98fk2" (OuterVolumeSpecName: "kube-api-access-98fk2") pod "f450e974-ffa7-479f-983b-71eaab8a1e3a" (UID: "f450e974-ffa7-479f-983b-71eaab8a1e3a"). InnerVolumeSpecName "kube-api-access-98fk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:23:32.755687 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:32.755598 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98fk2\" (UniqueName: \"kubernetes.io/projected/f450e974-ffa7-479f-983b-71eaab8a1e3a-kube-api-access-98fk2\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:23:33.435177 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:33.435145 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-140604917805392" Aug 04 13:23:33.435177 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:33.435170 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1fd3a2d7c9d405d40161c079caf575c5b36c72c3b361a44dd9acbb4832f755bc" Aug 04 13:23:33.445703 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:23:33.445669 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-140604917805392\" is forbidden: User \"system:node:ip-10-0-132-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-96.ec2.internal' and this object" podUID="f450e974-ffa7-479f-983b-71eaab8a1e3a" pod="opendatahub/test-curl-7696-140604917805392" Aug 04 13:23:34.277113 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:34.277071 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f450e974-ffa7-479f-983b-71eaab8a1e3a" path="/var/lib/kubelet/pods/f450e974-ffa7-479f-983b-71eaab8a1e3a/volumes" Aug 04 13:23:41.029798 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.029759 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4"] Aug 04 13:23:41.030222 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.030165 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f450e974-ffa7-479f-983b-71eaab8a1e3a" containerName="test-curl-7696-140604917805392" Aug 04 13:23:41.030222 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.030180 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f450e974-ffa7-479f-983b-71eaab8a1e3a" containerName="test-curl-7696-140604917805392" Aug 04 13:23:41.030302 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.030251 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f450e974-ffa7-479f-983b-71eaab8a1e3a" containerName="test-curl-7696-140604917805392" Aug 04 13:23:41.033460 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.033440 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.036119 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.036095 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-2ef922-serving-cert\"" Aug 04 13:23:41.043975 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.043945 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4"] Aug 04 13:23:41.129848 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.129805 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hql8n\" (UniqueName: \"kubernetes.io/projected/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-kube-api-access-hql8n\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.130012 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.129889 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.130012 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.129914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.231044 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.231004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.231044 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.231044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.231256 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.231115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hql8n\" (UniqueName: \"kubernetes.io/projected/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-kube-api-access-hql8n\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.231769 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.231741 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.233916 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.233894 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.241234 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.241213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hql8n\" (UniqueName: \"kubernetes.io/projected/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-kube-api-access-hql8n\") pod \"maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.345278 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.345240 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:41.485332 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:41.485306 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4"] Aug 04 13:23:41.487324 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:23:41.487300 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ab7fab_d1ef_4b0c_b482_874a120a2f7a.slice/crio-14ce49937837ca979919f3a0e2538028e45517bed13d015eab73d6b4f9e42198 WatchSource:0}: Error finding container 14ce49937837ca979919f3a0e2538028e45517bed13d015eab73d6b4f9e42198: Status 404 returned error can't find the container with id 14ce49937837ca979919f3a0e2538028e45517bed13d015eab73d6b4f9e42198 Aug 04 13:23:42.485691 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:42.485657 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" event={"ID":"59ab7fab-d1ef-4b0c-b482-874a120a2f7a","Type":"ContainerStarted","Data":"5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80"} Aug 04 13:23:42.486119 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:42.485707 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" event={"ID":"59ab7fab-d1ef-4b0c-b482-874a120a2f7a","Type":"ContainerStarted","Data":"14ce49937837ca979919f3a0e2538028e45517bed13d015eab73d6b4f9e42198"} Aug 04 13:23:42.486119 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:42.485781 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:23:42.505984 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:42.505934 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" podStartSLOduration=1.097918309 podStartE2EDuration="1.505918958s" totalImagesPullingTime="408.000649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:23:42.50398254 +0000 UTC m=+3382.740413913" watchObservedRunningTime="2026-08-04 13:23:42.505918958 +0000 UTC m=+3382.742350327" Aug 04 13:23:44.500353 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:44.500314 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:23:48.495845 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:23:48.495810 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:24:09.982226 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:09.982193 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd"] Aug 04 13:24:09.986110 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:09.986088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:09.988928 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:09.988906 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-77ce5c-serving-cert\"" Aug 04 13:24:10.000280 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.000253 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd"] Aug 04 13:24:10.092372 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.092331 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m8qn\" (UniqueName: \"kubernetes.io/projected/864d9693-b747-4508-bcbc-7f299ccb3b34-kube-api-access-7m8qn\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.092372 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.092375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864d9693-b747-4508-bcbc-7f299ccb3b34-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.092585 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.092433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864d9693-b747-4508-bcbc-7f299ccb3b34-maas-api-tls\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.193394 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.193350 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m8qn\" (UniqueName: \"kubernetes.io/projected/864d9693-b747-4508-bcbc-7f299ccb3b34-kube-api-access-7m8qn\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.193394 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.193393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864d9693-b747-4508-bcbc-7f299ccb3b34-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.193633 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.193513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864d9693-b747-4508-bcbc-7f299ccb3b34-maas-api-tls\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.194159 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.194140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864d9693-b747-4508-bcbc-7f299ccb3b34-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.196308 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.196287 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864d9693-b747-4508-bcbc-7f299ccb3b34-maas-api-tls\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.203735 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.203703 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m8qn\" (UniqueName: \"kubernetes.io/projected/864d9693-b747-4508-bcbc-7f299ccb3b34-kube-api-access-7m8qn\") pod \"maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.299769 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.299686 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:10.434476 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.434451 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd"] Aug 04 13:24:10.436314 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:24:10.436289 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864d9693_b747_4508_bcbc_7f299ccb3b34.slice/crio-beed76c8f5d24969008dee64bc761b523a4f5219beded7681542ff42b48a0880 WatchSource:0}: Error finding container beed76c8f5d24969008dee64bc761b523a4f5219beded7681542ff42b48a0880: Status 404 returned error can't find the container with id beed76c8f5d24969008dee64bc761b523a4f5219beded7681542ff42b48a0880 Aug 04 13:24:10.638612 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:10.638571 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" event={"ID":"864d9693-b747-4508-bcbc-7f299ccb3b34","Type":"ContainerStarted","Data":"beed76c8f5d24969008dee64bc761b523a4f5219beded7681542ff42b48a0880"} Aug 04 13:24:11.646480 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:11.646447 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" event={"ID":"864d9693-b747-4508-bcbc-7f299ccb3b34","Type":"ContainerStarted","Data":"215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38"} Aug 04 13:24:11.646943 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:11.646583 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:11.668983 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:11.668941 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" podStartSLOduration=2.085886744 podStartE2EDuration="2.66892523s" totalImagesPullingTime="583.038486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:24:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:24:11.667296702 +0000 UTC m=+3411.903728076" watchObservedRunningTime="2026-08-04 13:24:11.66892523 +0000 UTC m=+3411.905356600" Aug 04 13:24:14.203246 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:14.203207 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:24:17.658010 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:17.657980 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:24:46.908704 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:24:46.908664 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:25:19.504727 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:19.504691 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:25:35.379259 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.379220 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5"] Aug 04 13:25:35.382501 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.382484 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:35.385334 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.385313 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-b88718-serving-cert\"" Aug 04 13:25:35.394226 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.394204 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5"] Aug 04 13:25:35.520276 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.520242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:35.520276 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.520277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/953d4f99-11f5-43e1-9593-f3f86e4b0420-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:35.520487 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.520316 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbvmr\" (UniqueName: \"kubernetes.io/projected/953d4f99-11f5-43e1-9593-f3f86e4b0420-kube-api-access-mbvmr\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:35.621657 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.621619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbvmr\" (UniqueName: \"kubernetes.io/projected/953d4f99-11f5-43e1-9593-f3f86e4b0420-kube-api-access-mbvmr\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:35.621863 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.621698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:35.621863 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.621719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/953d4f99-11f5-43e1-9593-f3f86e4b0420-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:35.621962 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:25:35.621864 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-serving-cert: secret "maas-api-e2e-ipp-cleanup-b88718-serving-cert" not found Aug 04 13:25:35.621962 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:25:35.621947 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls podName:953d4f99-11f5-43e1-9593-f3f86e4b0420 nodeName:}" failed. No retries permitted until 2026-08-04 13:25:36.121929425 +0000 UTC m=+3496.358360773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" (UID: "953d4f99-11f5-43e1-9593-f3f86e4b0420") : secret "maas-api-e2e-ipp-cleanup-b88718-serving-cert" not found Aug 04 13:25:35.622351 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.622334 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/953d4f99-11f5-43e1-9593-f3f86e4b0420-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:35.631678 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:35.631613 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbvmr\" (UniqueName: \"kubernetes.io/projected/953d4f99-11f5-43e1-9593-f3f86e4b0420-kube-api-access-mbvmr\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:36.126738 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:36.126700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:36.129257 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:36.129236 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:36.294123 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:36.294093 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:36.439223 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:36.439187 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5"] Aug 04 13:25:36.441104 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:25:36.441076 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953d4f99_11f5_43e1_9593_f3f86e4b0420.slice/crio-20eac36f00c9543d9006d8abf336cd9f0568a7247fa1aa605f9515726d750a70 WatchSource:0}: Error finding container 20eac36f00c9543d9006d8abf336cd9f0568a7247fa1aa605f9515726d750a70: Status 404 returned error can't find the container with id 20eac36f00c9543d9006d8abf336cd9f0568a7247fa1aa605f9515726d750a70 Aug 04 13:25:37.121068 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:37.120953 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" event={"ID":"953d4f99-11f5-43e1-9593-f3f86e4b0420","Type":"ContainerStarted","Data":"8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635"} Aug 04 13:25:37.121068 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:37.121016 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" event={"ID":"953d4f99-11f5-43e1-9593-f3f86e4b0420","Type":"ContainerStarted","Data":"20eac36f00c9543d9006d8abf336cd9f0568a7247fa1aa605f9515726d750a70"} Aug 04 13:25:37.121068 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:37.121056 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:37.141468 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:37.141413 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" podStartSLOduration=1.750405297 podStartE2EDuration="2.141394806s" totalImagesPullingTime="390.989509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:25:37.139134845 +0000 UTC m=+3497.375566216" watchObservedRunningTime="2026-08-04 13:25:37.141394806 +0000 UTC m=+3497.377826222" Aug 04 13:25:41.092058 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:41.092021 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:25:43.132445 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:43.132364 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:25:59.943254 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:59.943214 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4"] Aug 04 13:25:59.947864 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:59.947823 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" Aug 04 13:25:59.951289 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:59.951272 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:25:59.959597 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:25:59.959575 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4"] Aug 04 13:26:00.145552 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:00.145496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06aff4eb-e7c8-4ea0-97a5-2f6a4b241323-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4\" (UID: \"06aff4eb-e7c8-4ea0-97a5-2f6a4b241323\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" Aug 04 13:26:00.246727 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:00.246636 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06aff4eb-e7c8-4ea0-97a5-2f6a4b241323-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4\" (UID: \"06aff4eb-e7c8-4ea0-97a5-2f6a4b241323\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" Aug 04 13:26:00.247246 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:00.247227 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06aff4eb-e7c8-4ea0-97a5-2f6a4b241323-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4\" (UID: \"06aff4eb-e7c8-4ea0-97a5-2f6a4b241323\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" Aug 04 13:26:00.258614 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:00.258592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" Aug 04 13:26:00.400224 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:00.400197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4"] Aug 04 13:26:00.403518 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:26:00.403486 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06aff4eb_e7c8_4ea0_97a5_2f6a4b241323.slice/crio-3999de93e3311ba218c4410b81100cce92c0ec7748dbe347179534eb5abb4c0a WatchSource:0}: Error finding container 3999de93e3311ba218c4410b81100cce92c0ec7748dbe347179534eb5abb4c0a: Status 404 returned error can't find the container with id 3999de93e3311ba218c4410b81100cce92c0ec7748dbe347179534eb5abb4c0a Aug 04 13:26:01.253342 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:01.253311 2580 generic.go:356] "Generic (PLEG): container finished" podID="06aff4eb-e7c8-4ea0-97a5-2f6a4b241323" containerID="a99a3cf0061dab17cda06fa85fffebab7bd3391b6c816c3e98ab15c6bc987acf" exitCode=0 Aug 04 13:26:01.253777 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:01.253403 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" event={"ID":"06aff4eb-e7c8-4ea0-97a5-2f6a4b241323","Type":"ContainerDied","Data":"a99a3cf0061dab17cda06fa85fffebab7bd3391b6c816c3e98ab15c6bc987acf"} Aug 04 13:26:01.253777 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:01.253455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" event={"ID":"06aff4eb-e7c8-4ea0-97a5-2f6a4b241323","Type":"ContainerStarted","Data":"3999de93e3311ba218c4410b81100cce92c0ec7748dbe347179534eb5abb4c0a"} Aug 04 13:26:02.396904 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:02.396876 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" Aug 04 13:26:02.462413 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:02.462375 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06aff4eb-e7c8-4ea0-97a5-2f6a4b241323-maas-api-service-ca\") pod \"06aff4eb-e7c8-4ea0-97a5-2f6a4b241323\" (UID: \"06aff4eb-e7c8-4ea0-97a5-2f6a4b241323\") " Aug 04 13:26:02.462740 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:02.462716 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06aff4eb-e7c8-4ea0-97a5-2f6a4b241323-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06aff4eb-e7c8-4ea0-97a5-2f6a4b241323" (UID: "06aff4eb-e7c8-4ea0-97a5-2f6a4b241323"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:26:02.563165 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:02.563133 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06aff4eb-e7c8-4ea0-97a5-2f6a4b241323-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:26:03.265062 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:03.265027 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" Aug 04 13:26:03.265236 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:03.265022 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4" event={"ID":"06aff4eb-e7c8-4ea0-97a5-2f6a4b241323","Type":"ContainerDied","Data":"3999de93e3311ba218c4410b81100cce92c0ec7748dbe347179534eb5abb4c0a"} Aug 04 13:26:03.265236 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:03.265153 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3999de93e3311ba218c4410b81100cce92c0ec7748dbe347179534eb5abb4c0a" Aug 04 13:26:05.973486 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:05.973451 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5"] Aug 04 13:26:05.973993 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:05.973650 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" podUID="953d4f99-11f5-43e1-9593-f3f86e4b0420" containerName="maas-api" containerID="cri-o://8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635" gracePeriod=30 Aug 04 13:26:06.206418 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.206387 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4"] Aug 04 13:26:06.208696 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.208669 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b88718-zhlr4"] Aug 04 13:26:06.231928 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.231861 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:26:06.277535 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.277500 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06aff4eb-e7c8-4ea0-97a5-2f6a4b241323" path="/var/lib/kubelet/pods/06aff4eb-e7c8-4ea0-97a5-2f6a4b241323/volumes" Aug 04 13:26:06.281652 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.281624 2580 generic.go:356] "Generic (PLEG): container finished" podID="953d4f99-11f5-43e1-9593-f3f86e4b0420" containerID="8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635" exitCode=0 Aug 04 13:26:06.281804 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.281656 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" Aug 04 13:26:06.281804 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.281705 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" event={"ID":"953d4f99-11f5-43e1-9593-f3f86e4b0420","Type":"ContainerDied","Data":"8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635"} Aug 04 13:26:06.281804 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.281742 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5" event={"ID":"953d4f99-11f5-43e1-9593-f3f86e4b0420","Type":"ContainerDied","Data":"20eac36f00c9543d9006d8abf336cd9f0568a7247fa1aa605f9515726d750a70"} Aug 04 13:26:06.281804 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.281764 2580 scope.go:117] "RemoveContainer" containerID="8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635" Aug 04 13:26:06.289676 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.289653 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls\") pod \"953d4f99-11f5-43e1-9593-f3f86e4b0420\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " Aug 04 13:26:06.289800 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.289721 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbvmr\" (UniqueName: \"kubernetes.io/projected/953d4f99-11f5-43e1-9593-f3f86e4b0420-kube-api-access-mbvmr\") pod \"953d4f99-11f5-43e1-9593-f3f86e4b0420\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " Aug 04 13:26:06.289800 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.289775 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/953d4f99-11f5-43e1-9593-f3f86e4b0420-openshift-service-ca\") pod \"953d4f99-11f5-43e1-9593-f3f86e4b0420\" (UID: \"953d4f99-11f5-43e1-9593-f3f86e4b0420\") " Aug 04 13:26:06.290171 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.290145 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953d4f99-11f5-43e1-9593-f3f86e4b0420-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "953d4f99-11f5-43e1-9593-f3f86e4b0420" (UID: "953d4f99-11f5-43e1-9593-f3f86e4b0420"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:26:06.291540 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.291521 2580 scope.go:117] "RemoveContainer" containerID="8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635" Aug 04 13:26:06.291863 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.291822 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "953d4f99-11f5-43e1-9593-f3f86e4b0420" (UID: "953d4f99-11f5-43e1-9593-f3f86e4b0420"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 13:26:06.291935 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:26:06.291824 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635\": container with ID starting with 8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635 not found: ID does not exist" containerID="8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635" Aug 04 13:26:06.291935 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.291890 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635"} err="failed to get container status \"8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635\": rpc error: code = NotFound desc = could not find container \"8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635\": container with ID starting with 8085e1f4dd6fc669b633811351ff929a6289f2c0e73a31022cbc015ab3255635 not found: ID does not exist" Aug 04 13:26:06.292052 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.292038 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953d4f99-11f5-43e1-9593-f3f86e4b0420-kube-api-access-mbvmr" (OuterVolumeSpecName: "kube-api-access-mbvmr") pod "953d4f99-11f5-43e1-9593-f3f86e4b0420" (UID: "953d4f99-11f5-43e1-9593-f3f86e4b0420"). InnerVolumeSpecName "kube-api-access-mbvmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:26:06.390859 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.390774 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/953d4f99-11f5-43e1-9593-f3f86e4b0420-openshift-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:26:06.390859 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.390825 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/953d4f99-11f5-43e1-9593-f3f86e4b0420-maas-api-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:26:06.390859 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.390859 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbvmr\" (UniqueName: \"kubernetes.io/projected/953d4f99-11f5-43e1-9593-f3f86e4b0420-kube-api-access-mbvmr\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:26:06.606735 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.606701 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5"] Aug 04 13:26:06.611194 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:06.611166 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b88718-74c5964bc8-4c9n5"] Aug 04 13:26:08.277085 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:08.277045 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="953d4f99-11f5-43e1-9593-f3f86e4b0420" path="/var/lib/kubelet/pods/953d4f99-11f5-43e1-9593-f3f86e4b0420/volumes" Aug 04 13:26:09.398765 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:09.398728 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:26:17.006576 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.006536 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd"] Aug 04 13:26:17.006987 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.006968 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06aff4eb-e7c8-4ea0-97a5-2f6a4b241323" containerName="revoke-keys" Aug 04 13:26:17.006987 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.006980 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="06aff4eb-e7c8-4ea0-97a5-2f6a4b241323" containerName="revoke-keys" Aug 04 13:26:17.007322 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.006998 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953d4f99-11f5-43e1-9593-f3f86e4b0420" containerName="maas-api" Aug 04 13:26:17.007322 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.007004 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="953d4f99-11f5-43e1-9593-f3f86e4b0420" containerName="maas-api" Aug 04 13:26:17.007322 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.007067 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="06aff4eb-e7c8-4ea0-97a5-2f6a4b241323" containerName="revoke-keys" Aug 04 13:26:17.007322 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.007076 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="953d4f99-11f5-43e1-9593-f3f86e4b0420" containerName="maas-api" Aug 04 13:26:17.011536 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.011520 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" Aug 04 13:26:17.019170 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.019151 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:26:17.056577 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.056545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd"] Aug 04 13:26:17.086565 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.086535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd\" (UID: \"2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" Aug 04 13:26:17.187004 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.186970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd\" (UID: \"2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" Aug 04 13:26:17.187541 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.187522 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd\" (UID: \"2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" Aug 04 13:26:17.322168 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.322138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" Aug 04 13:26:17.461894 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:17.461865 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd"] Aug 04 13:26:17.464089 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:26:17.464058 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6a71f3_f2f7_4d2f_94a6_7d3b4b52cf42.slice/crio-ae0cb2aa7c841f04c7dac0c0efd595aa6d162b186df5a116793c78bde94a84d0 WatchSource:0}: Error finding container ae0cb2aa7c841f04c7dac0c0efd595aa6d162b186df5a116793c78bde94a84d0: Status 404 returned error can't find the container with id ae0cb2aa7c841f04c7dac0c0efd595aa6d162b186df5a116793c78bde94a84d0 Aug 04 13:26:18.350072 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:18.350037 2580 generic.go:356] "Generic (PLEG): container finished" podID="2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42" containerID="9c145e057e1aa0cbfc6d28ebab595ff5beadc433e0a24e9c317281e85d38aa19" exitCode=0 Aug 04 13:26:18.350498 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:18.350135 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" event={"ID":"2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42","Type":"ContainerDied","Data":"9c145e057e1aa0cbfc6d28ebab595ff5beadc433e0a24e9c317281e85d38aa19"} Aug 04 13:26:18.350498 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:18.350198 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" event={"ID":"2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42","Type":"ContainerStarted","Data":"ae0cb2aa7c841f04c7dac0c0efd595aa6d162b186df5a116793c78bde94a84d0"} Aug 04 13:26:19.489989 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:19.489962 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" Aug 04 13:26:19.607656 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:19.607539 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42-maas-api-service-ca\") pod \"2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42\" (UID: \"2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42\") " Aug 04 13:26:19.607952 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:19.607931 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42" (UID: "2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:26:19.709064 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:19.709012 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:26:20.363047 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:20.363006 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" event={"ID":"2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42","Type":"ContainerDied","Data":"ae0cb2aa7c841f04c7dac0c0efd595aa6d162b186df5a116793c78bde94a84d0"} Aug 04 13:26:20.363047 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:20.363047 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd" Aug 04 13:26:20.363292 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:20.363061 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae0cb2aa7c841f04c7dac0c0efd595aa6d162b186df5a116793c78bde94a84d0" Aug 04 13:26:30.607433 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:30.607398 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:26:32.720593 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:32.720547 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4"] Aug 04 13:26:32.721099 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:32.720801 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" podUID="59ab7fab-d1ef-4b0c-b482-874a120a2f7a" containerName="maas-api" containerID="cri-o://5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80" gracePeriod=30 Aug 04 13:26:32.949300 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:32.949266 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd"] Aug 04 13:26:32.951638 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:32.951609 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ef922-kzwmd"] Aug 04 13:26:32.979261 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:32.979193 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:26:33.123424 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.123393 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-maas-api-tls\") pod \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " Aug 04 13:26:33.123647 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.123536 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hql8n\" (UniqueName: \"kubernetes.io/projected/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-kube-api-access-hql8n\") pod \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " Aug 04 13:26:33.123647 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.123574 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-openshift-service-ca\") pod \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\" (UID: \"59ab7fab-d1ef-4b0c-b482-874a120a2f7a\") " Aug 04 13:26:33.124064 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.124029 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "59ab7fab-d1ef-4b0c-b482-874a120a2f7a" (UID: "59ab7fab-d1ef-4b0c-b482-874a120a2f7a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:26:33.125742 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.125714 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "59ab7fab-d1ef-4b0c-b482-874a120a2f7a" (UID: "59ab7fab-d1ef-4b0c-b482-874a120a2f7a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 13:26:33.125878 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.125818 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-kube-api-access-hql8n" (OuterVolumeSpecName: "kube-api-access-hql8n") pod "59ab7fab-d1ef-4b0c-b482-874a120a2f7a" (UID: "59ab7fab-d1ef-4b0c-b482-874a120a2f7a"). InnerVolumeSpecName "kube-api-access-hql8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:26:33.225313 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.225270 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-maas-api-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:26:33.225313 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.225308 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hql8n\" (UniqueName: \"kubernetes.io/projected/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-kube-api-access-hql8n\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:26:33.225313 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.225320 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ab7fab-d1ef-4b0c-b482-874a120a2f7a-openshift-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:26:33.437949 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.437919 2580 generic.go:356] "Generic (PLEG): container finished" podID="59ab7fab-d1ef-4b0c-b482-874a120a2f7a" containerID="5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80" exitCode=0 Aug 04 13:26:33.438161 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.437967 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" Aug 04 13:26:33.438161 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.437981 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" event={"ID":"59ab7fab-d1ef-4b0c-b482-874a120a2f7a","Type":"ContainerDied","Data":"5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80"} Aug 04 13:26:33.438161 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.438025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4" event={"ID":"59ab7fab-d1ef-4b0c-b482-874a120a2f7a","Type":"ContainerDied","Data":"14ce49937837ca979919f3a0e2538028e45517bed13d015eab73d6b4f9e42198"} Aug 04 13:26:33.438161 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.438046 2580 scope.go:117] "RemoveContainer" containerID="5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80" Aug 04 13:26:33.447814 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.447794 2580 scope.go:117] "RemoveContainer" containerID="5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80" Aug 04 13:26:33.448137 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:26:33.448117 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80\": container with ID starting with 5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80 not found: ID does not exist" containerID="5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80" Aug 04 13:26:33.448181 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.448147 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80"} err="failed to get container status \"5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80\": rpc error: code = NotFound desc = could not find container \"5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80\": container with ID starting with 5f0358ddd7c1d6348a3224c951685854c9fdfa08727cc7e7fe3c06cfe41aee80 not found: ID does not exist" Aug 04 13:26:33.461350 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.461317 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4"] Aug 04 13:26:33.466929 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:33.466897 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2ef922-6d886568fc-t6pr4"] Aug 04 13:26:34.277476 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:34.277438 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42" path="/var/lib/kubelet/pods/2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42/volumes" Aug 04 13:26:34.277876 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:34.277801 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59ab7fab-d1ef-4b0c-b482-874a120a2f7a" path="/var/lib/kubelet/pods/59ab7fab-d1ef-4b0c-b482-874a120a2f7a/volumes" Aug 04 13:26:39.695332 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:26:39.695291 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:27:20.571261 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:20.571234 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:27:20.582955 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:20.582932 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:27:23.827084 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.827048 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj"] Aug 04 13:27:23.827560 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.827543 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42" containerName="revoke-keys" Aug 04 13:27:23.827613 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.827563 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42" containerName="revoke-keys" Aug 04 13:27:23.827613 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.827580 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59ab7fab-d1ef-4b0c-b482-874a120a2f7a" containerName="maas-api" Aug 04 13:27:23.827613 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.827586 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="59ab7fab-d1ef-4b0c-b482-874a120a2f7a" containerName="maas-api" Aug 04 13:27:23.827735 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.827646 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="59ab7fab-d1ef-4b0c-b482-874a120a2f7a" containerName="maas-api" Aug 04 13:27:23.827735 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.827654 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c6a71f3-f2f7-4d2f-94a6-7d3b4b52cf42" containerName="revoke-keys" Aug 04 13:27:23.832119 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.832100 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" Aug 04 13:27:23.836250 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.836229 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:27:23.841398 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.841376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj"] Aug 04 13:27:23.847379 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.847356 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f765674-d34e-4615-88c3-4fbde9d4986d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj\" (UID: \"0f765674-d34e-4615-88c3-4fbde9d4986d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" Aug 04 13:27:23.947797 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.947755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f765674-d34e-4615-88c3-4fbde9d4986d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj\" (UID: \"0f765674-d34e-4615-88c3-4fbde9d4986d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" Aug 04 13:27:23.948470 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:23.948443 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f765674-d34e-4615-88c3-4fbde9d4986d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj\" (UID: \"0f765674-d34e-4615-88c3-4fbde9d4986d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" Aug 04 13:27:24.145340 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:24.145257 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" Aug 04 13:27:24.294892 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:24.294866 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj"] Aug 04 13:27:24.296002 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:27:24.295970 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f765674_d34e_4615_88c3_4fbde9d4986d.slice/crio-99486142c7ce0bfac065ee295004f9a8137802ea6342446b714335646bbc5f8c WatchSource:0}: Error finding container 99486142c7ce0bfac065ee295004f9a8137802ea6342446b714335646bbc5f8c: Status 404 returned error can't find the container with id 99486142c7ce0bfac065ee295004f9a8137802ea6342446b714335646bbc5f8c Aug 04 13:27:24.722021 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:24.721982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" event={"ID":"0f765674-d34e-4615-88c3-4fbde9d4986d","Type":"ContainerStarted","Data":"daa6174c35b4f7039307d618c0b0997be27181c9853d8d8f4e19b74723c42060"} Aug 04 13:27:24.722226 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:24.722040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" event={"ID":"0f765674-d34e-4615-88c3-4fbde9d4986d","Type":"ContainerStarted","Data":"99486142c7ce0bfac065ee295004f9a8137802ea6342446b714335646bbc5f8c"} Aug 04 13:27:24.746153 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:24.746051 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" podStartSLOduration=1.74603525 podStartE2EDuration="1.74603525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:27:24.745922937 +0000 UTC m=+3604.982354306" watchObservedRunningTime="2026-08-04 13:27:24.74603525 +0000 UTC m=+3604.982466620" Aug 04 13:27:25.729643 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:25.729603 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f765674-d34e-4615-88c3-4fbde9d4986d" containerID="daa6174c35b4f7039307d618c0b0997be27181c9853d8d8f4e19b74723c42060" exitCode=0 Aug 04 13:27:25.730064 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:25.729696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" event={"ID":"0f765674-d34e-4615-88c3-4fbde9d4986d","Type":"ContainerDied","Data":"daa6174c35b4f7039307d618c0b0997be27181c9853d8d8f4e19b74723c42060"} Aug 04 13:27:26.869505 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:26.869478 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" Aug 04 13:27:26.973621 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:26.973588 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f765674-d34e-4615-88c3-4fbde9d4986d-maas-api-service-ca\") pod \"0f765674-d34e-4615-88c3-4fbde9d4986d\" (UID: \"0f765674-d34e-4615-88c3-4fbde9d4986d\") " Aug 04 13:27:26.973975 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:26.973954 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f765674-d34e-4615-88c3-4fbde9d4986d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f765674-d34e-4615-88c3-4fbde9d4986d" (UID: "0f765674-d34e-4615-88c3-4fbde9d4986d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:27:27.075122 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:27.075091 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f765674-d34e-4615-88c3-4fbde9d4986d-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:27.741542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:27.741513 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" Aug 04 13:27:27.741542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:27.741538 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj" event={"ID":"0f765674-d34e-4615-88c3-4fbde9d4986d","Type":"ContainerDied","Data":"99486142c7ce0bfac065ee295004f9a8137802ea6342446b714335646bbc5f8c"} Aug 04 13:27:27.741767 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:27.741586 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99486142c7ce0bfac065ee295004f9a8137802ea6342446b714335646bbc5f8c" Aug 04 13:27:29.278927 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.278892 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd"] Aug 04 13:27:29.279357 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.279129 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" podUID="864d9693-b747-4508-bcbc-7f299ccb3b34" containerName="maas-api" containerID="cri-o://215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38" gracePeriod=30 Aug 04 13:27:29.529038 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.528968 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:27:29.545686 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.545653 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj"] Aug 04 13:27:29.554953 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.554923 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-77ce5c-twnkj"] Aug 04 13:27:29.595693 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.595656 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864d9693-b747-4508-bcbc-7f299ccb3b34-maas-api-tls\") pod \"864d9693-b747-4508-bcbc-7f299ccb3b34\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " Aug 04 13:27:29.595884 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.595715 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864d9693-b747-4508-bcbc-7f299ccb3b34-openshift-service-ca\") pod \"864d9693-b747-4508-bcbc-7f299ccb3b34\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " Aug 04 13:27:29.595884 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.595775 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7m8qn\" (UniqueName: \"kubernetes.io/projected/864d9693-b747-4508-bcbc-7f299ccb3b34-kube-api-access-7m8qn\") pod \"864d9693-b747-4508-bcbc-7f299ccb3b34\" (UID: \"864d9693-b747-4508-bcbc-7f299ccb3b34\") " Aug 04 13:27:29.596183 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.596158 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/864d9693-b747-4508-bcbc-7f299ccb3b34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "864d9693-b747-4508-bcbc-7f299ccb3b34" (UID: "864d9693-b747-4508-bcbc-7f299ccb3b34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:27:29.598003 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.597981 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/864d9693-b747-4508-bcbc-7f299ccb3b34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "864d9693-b747-4508-bcbc-7f299ccb3b34" (UID: "864d9693-b747-4508-bcbc-7f299ccb3b34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 13:27:29.598098 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.598004 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/864d9693-b747-4508-bcbc-7f299ccb3b34-kube-api-access-7m8qn" (OuterVolumeSpecName: "kube-api-access-7m8qn") pod "864d9693-b747-4508-bcbc-7f299ccb3b34" (UID: "864d9693-b747-4508-bcbc-7f299ccb3b34"). InnerVolumeSpecName "kube-api-access-7m8qn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:27:29.696656 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.696618 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m8qn\" (UniqueName: \"kubernetes.io/projected/864d9693-b747-4508-bcbc-7f299ccb3b34-kube-api-access-7m8qn\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:29.696656 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.696650 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864d9693-b747-4508-bcbc-7f299ccb3b34-maas-api-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:29.696656 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.696666 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864d9693-b747-4508-bcbc-7f299ccb3b34-openshift-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:29.757917 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.757880 2580 generic.go:356] "Generic (PLEG): container finished" podID="864d9693-b747-4508-bcbc-7f299ccb3b34" containerID="215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38" exitCode=0 Aug 04 13:27:29.758108 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.757934 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" Aug 04 13:27:29.758108 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.757966 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" event={"ID":"864d9693-b747-4508-bcbc-7f299ccb3b34","Type":"ContainerDied","Data":"215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38"} Aug 04 13:27:29.758108 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.758022 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd" event={"ID":"864d9693-b747-4508-bcbc-7f299ccb3b34","Type":"ContainerDied","Data":"beed76c8f5d24969008dee64bc761b523a4f5219beded7681542ff42b48a0880"} Aug 04 13:27:29.758108 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.758046 2580 scope.go:117] "RemoveContainer" containerID="215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38" Aug 04 13:27:29.768103 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.768083 2580 scope.go:117] "RemoveContainer" containerID="215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38" Aug 04 13:27:29.768365 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:27:29.768343 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38\": container with ID starting with 215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38 not found: ID does not exist" containerID="215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38" Aug 04 13:27:29.768451 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.768375 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38"} err="failed to get container status \"215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38\": rpc error: code = NotFound desc = could not find container \"215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38\": container with ID starting with 215abb39a198426bdca4faea7129eac288792a34099f52124125a984889c1f38 not found: ID does not exist" Aug 04 13:27:29.783535 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.783473 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd"] Aug 04 13:27:29.790918 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:29.790884 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-77ce5c-5b9486ddd8-kkwwd"] Aug 04 13:27:30.277852 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:30.277791 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f765674-d34e-4615-88c3-4fbde9d4986d" path="/var/lib/kubelet/pods/0f765674-d34e-4615-88c3-4fbde9d4986d/volumes" Aug 04 13:27:30.278261 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:30.278246 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="864d9693-b747-4508-bcbc-7f299ccb3b34" path="/var/lib/kubelet/pods/864d9693-b747-4508-bcbc-7f299ccb3b34/volumes" Aug 04 13:27:31.761692 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:31.761644 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:27:37.818126 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.818086 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv"] Aug 04 13:27:37.818542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.818427 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="864d9693-b747-4508-bcbc-7f299ccb3b34" containerName="maas-api" Aug 04 13:27:37.818542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.818438 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="864d9693-b747-4508-bcbc-7f299ccb3b34" containerName="maas-api" Aug 04 13:27:37.818542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.818448 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f765674-d34e-4615-88c3-4fbde9d4986d" containerName="revoke-keys" Aug 04 13:27:37.818542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.818454 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f765674-d34e-4615-88c3-4fbde9d4986d" containerName="revoke-keys" Aug 04 13:27:37.818542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.818514 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f765674-d34e-4615-88c3-4fbde9d4986d" containerName="revoke-keys" Aug 04 13:27:37.818542 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.818522 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="864d9693-b747-4508-bcbc-7f299ccb3b34" containerName="maas-api" Aug 04 13:27:37.821724 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.821708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" Aug 04 13:27:37.824576 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.824555 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:27:37.831214 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.831192 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv"] Aug 04 13:27:37.864952 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.864915 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2fe95-44d6-4e89-85b0-0d882b2fb335-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv\" (UID: \"1ce2fe95-44d6-4e89-85b0-0d882b2fb335\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" Aug 04 13:27:37.966134 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.966099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2fe95-44d6-4e89-85b0-0d882b2fb335-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv\" (UID: \"1ce2fe95-44d6-4e89-85b0-0d882b2fb335\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" Aug 04 13:27:37.966729 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:37.966708 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2fe95-44d6-4e89-85b0-0d882b2fb335-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv\" (UID: \"1ce2fe95-44d6-4e89-85b0-0d882b2fb335\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" Aug 04 13:27:38.132858 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:38.132750 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" Aug 04 13:27:38.276844 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:27:38.276792 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce2fe95_44d6_4e89_85b0_0d882b2fb335.slice/crio-ab3dce3fcf687ac80d18dcdb1023d8afdc42a5d717dbee42fcf1a35b2e821506 WatchSource:0}: Error finding container ab3dce3fcf687ac80d18dcdb1023d8afdc42a5d717dbee42fcf1a35b2e821506: Status 404 returned error can't find the container with id ab3dce3fcf687ac80d18dcdb1023d8afdc42a5d717dbee42fcf1a35b2e821506 Aug 04 13:27:38.279130 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:38.279109 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv"] Aug 04 13:27:38.810900 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:38.810863 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" event={"ID":"1ce2fe95-44d6-4e89-85b0-0d882b2fb335","Type":"ContainerStarted","Data":"83a00c7da8407e6ecb8662fa8bddeea27d8dc993a724f366c3bf6179294d715c"} Aug 04 13:27:38.811082 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:38.810959 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" event={"ID":"1ce2fe95-44d6-4e89-85b0-0d882b2fb335","Type":"ContainerStarted","Data":"ab3dce3fcf687ac80d18dcdb1023d8afdc42a5d717dbee42fcf1a35b2e821506"} Aug 04 13:27:38.828792 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:38.828745 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" podStartSLOduration=1.828731888 podStartE2EDuration="1.828731888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 13:27:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:27:38.82796223 +0000 UTC m=+3619.064393601" watchObservedRunningTime="2026-08-04 13:27:38.828731888 +0000 UTC m=+3619.065163258" Aug 04 13:27:39.818264 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:39.818231 2580 generic.go:356] "Generic (PLEG): container finished" podID="1ce2fe95-44d6-4e89-85b0-0d882b2fb335" containerID="83a00c7da8407e6ecb8662fa8bddeea27d8dc993a724f366c3bf6179294d715c" exitCode=0 Aug 04 13:27:39.818440 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:39.818281 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" event={"ID":"1ce2fe95-44d6-4e89-85b0-0d882b2fb335","Type":"ContainerDied","Data":"83a00c7da8407e6ecb8662fa8bddeea27d8dc993a724f366c3bf6179294d715c"} Aug 04 13:27:40.967295 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:40.967267 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" Aug 04 13:27:41.093196 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:41.093087 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2fe95-44d6-4e89-85b0-0d882b2fb335-maas-api-service-ca\") pod \"1ce2fe95-44d6-4e89-85b0-0d882b2fb335\" (UID: \"1ce2fe95-44d6-4e89-85b0-0d882b2fb335\") " Aug 04 13:27:41.093459 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:41.093433 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ce2fe95-44d6-4e89-85b0-0d882b2fb335-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ce2fe95-44d6-4e89-85b0-0d882b2fb335" (UID: "1ce2fe95-44d6-4e89-85b0-0d882b2fb335"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:27:41.194311 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:41.194267 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2fe95-44d6-4e89-85b0-0d882b2fb335-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:41.831299 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:41.831266 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" Aug 04 13:27:41.831299 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:41.831294 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv" event={"ID":"1ce2fe95-44d6-4e89-85b0-0d882b2fb335","Type":"ContainerDied","Data":"ab3dce3fcf687ac80d18dcdb1023d8afdc42a5d717dbee42fcf1a35b2e821506"} Aug 04 13:27:41.831519 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:41.831342 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab3dce3fcf687ac80d18dcdb1023d8afdc42a5d717dbee42fcf1a35b2e821506" Aug 04 13:27:48.430974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.430000 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk"] Aug 04 13:27:48.430974 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.430282 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" podUID="d342e2df-f388-45ed-9f76-745b8d1a46a7" containerName="maas-api" containerID="cri-o://a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f" gracePeriod=30 Aug 04 13:27:48.466091 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.466055 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw"] Aug 04 13:27:48.468235 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.468213 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6a00c1-29764155-dfcdw"] Aug 04 13:27:48.485912 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.485874 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" podUID="d342e2df-f388-45ed-9f76-745b8d1a46a7" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.53:8443/health\": dial tcp 10.132.0.53:8443: connect: connection refused" Aug 04 13:27:48.687403 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.687336 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:27:48.750488 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.750444 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv"] Aug 04 13:27:48.754194 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.754164 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6a00c1-6ljlv"] Aug 04 13:27:48.755768 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.755753 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d342e2df-f388-45ed-9f76-745b8d1a46a7-openshift-service-ca\") pod \"d342e2df-f388-45ed-9f76-745b8d1a46a7\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " Aug 04 13:27:48.755860 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.755786 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls\") pod \"d342e2df-f388-45ed-9f76-745b8d1a46a7\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " Aug 04 13:27:48.755860 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.755852 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8wg7\" (UniqueName: \"kubernetes.io/projected/d342e2df-f388-45ed-9f76-745b8d1a46a7-kube-api-access-s8wg7\") pod \"d342e2df-f388-45ed-9f76-745b8d1a46a7\" (UID: \"d342e2df-f388-45ed-9f76-745b8d1a46a7\") " Aug 04 13:27:48.756159 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.756134 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d342e2df-f388-45ed-9f76-745b8d1a46a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d342e2df-f388-45ed-9f76-745b8d1a46a7" (UID: "d342e2df-f388-45ed-9f76-745b8d1a46a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:27:48.758076 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.758047 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d342e2df-f388-45ed-9f76-745b8d1a46a7" (UID: "d342e2df-f388-45ed-9f76-745b8d1a46a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 13:27:48.758160 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.758071 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d342e2df-f388-45ed-9f76-745b8d1a46a7-kube-api-access-s8wg7" (OuterVolumeSpecName: "kube-api-access-s8wg7") pod "d342e2df-f388-45ed-9f76-745b8d1a46a7" (UID: "d342e2df-f388-45ed-9f76-745b8d1a46a7"). InnerVolumeSpecName "kube-api-access-s8wg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:27:48.857458 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.857415 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d342e2df-f388-45ed-9f76-745b8d1a46a7-openshift-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:48.857458 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.857450 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d342e2df-f388-45ed-9f76-745b8d1a46a7-maas-api-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:48.857458 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.857460 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8wg7\" (UniqueName: \"kubernetes.io/projected/d342e2df-f388-45ed-9f76-745b8d1a46a7-kube-api-access-s8wg7\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:48.872896 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.872855 2580 generic.go:356] "Generic (PLEG): container finished" podID="d342e2df-f388-45ed-9f76-745b8d1a46a7" containerID="a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f" exitCode=0 Aug 04 13:27:48.872896 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.872901 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" event={"ID":"d342e2df-f388-45ed-9f76-745b8d1a46a7","Type":"ContainerDied","Data":"a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f"} Aug 04 13:27:48.873106 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.872944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" event={"ID":"d342e2df-f388-45ed-9f76-745b8d1a46a7","Type":"ContainerDied","Data":"65c93b6a5859d42abb3ac3882709b67c5a3c4469cda3385f2ee01d23d0e79fe5"} Aug 04 13:27:48.873106 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.872965 2580 scope.go:117] "RemoveContainer" containerID="a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f" Aug 04 13:27:48.873106 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.872901 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk" Aug 04 13:27:48.883008 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.882988 2580 scope.go:117] "RemoveContainer" containerID="a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f" Aug 04 13:27:48.883310 ip-10-0-132-96 kubenswrapper[2580]: E0804 13:27:48.883289 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f\": container with ID starting with a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f not found: ID does not exist" containerID="a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f" Aug 04 13:27:48.883375 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.883319 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f"} err="failed to get container status \"a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f\": rpc error: code = NotFound desc = could not find container \"a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f\": container with ID starting with a5266b408e65b5fcc6e62578aef379a47637035e8dd46e4c3932def1fb18111f not found: ID does not exist" Aug 04 13:27:48.914411 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.914377 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk"] Aug 04 13:27:48.921891 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:48.921853 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6a00c1-687cfd95b4-t9nzk"] Aug 04 13:27:50.277915 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:50.277885 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ce2fe95-44d6-4e89-85b0-0d882b2fb335" path="/var/lib/kubelet/pods/1ce2fe95-44d6-4e89-85b0-0d882b2fb335/volumes" Aug 04 13:27:50.278374 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:50.278163 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764" path="/var/lib/kubelet/pods/54bcac1f-e1d9-4ed4-a6e6-027b6b2c0764/volumes" Aug 04 13:27:50.278466 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:50.278454 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d342e2df-f388-45ed-9f76-745b8d1a46a7" path="/var/lib/kubelet/pods/d342e2df-f388-45ed-9f76-745b8d1a46a7/volumes" Aug 04 13:27:50.750868 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:50.750812 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:27:56.324705 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.324659 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2"] Aug 04 13:27:56.325240 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.325203 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ce2fe95-44d6-4e89-85b0-0d882b2fb335" containerName="revoke-keys" Aug 04 13:27:56.325240 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.325221 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ce2fe95-44d6-4e89-85b0-0d882b2fb335" containerName="revoke-keys" Aug 04 13:27:56.325359 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.325246 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d342e2df-f388-45ed-9f76-745b8d1a46a7" containerName="maas-api" Aug 04 13:27:56.325359 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.325255 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d342e2df-f388-45ed-9f76-745b8d1a46a7" containerName="maas-api" Aug 04 13:27:56.325464 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.325358 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d342e2df-f388-45ed-9f76-745b8d1a46a7" containerName="maas-api" Aug 04 13:27:56.325464 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.325376 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ce2fe95-44d6-4e89-85b0-0d882b2fb335" containerName="revoke-keys" Aug 04 13:27:56.330019 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.329995 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" Aug 04 13:27:56.332922 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.332899 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sbvbg\"" Aug 04 13:27:56.337796 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.337750 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2"] Aug 04 13:27:56.425911 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.425852 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c0895f-e804-4682-b6c8-44f1702e7408-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2\" (UID: \"e3c0895f-e804-4682-b6c8-44f1702e7408\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" Aug 04 13:27:56.526613 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.526571 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c0895f-e804-4682-b6c8-44f1702e7408-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2\" (UID: \"e3c0895f-e804-4682-b6c8-44f1702e7408\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" Aug 04 13:27:56.527234 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.527212 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c0895f-e804-4682-b6c8-44f1702e7408-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2\" (UID: \"e3c0895f-e804-4682-b6c8-44f1702e7408\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" Aug 04 13:27:56.641743 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.641656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" Aug 04 13:27:56.783213 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.783184 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2"] Aug 04 13:27:56.785145 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:27:56.785115 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c0895f_e804_4682_b6c8_44f1702e7408.slice/crio-6cc30bf7735db8980ad78814b1690b09251e0e14b789d0ded687b02aaabbabd5 WatchSource:0}: Error finding container 6cc30bf7735db8980ad78814b1690b09251e0e14b789d0ded687b02aaabbabd5: Status 404 returned error can't find the container with id 6cc30bf7735db8980ad78814b1690b09251e0e14b789d0ded687b02aaabbabd5 Aug 04 13:27:56.921189 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:56.921095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" event={"ID":"e3c0895f-e804-4682-b6c8-44f1702e7408","Type":"ContainerStarted","Data":"6cc30bf7735db8980ad78814b1690b09251e0e14b789d0ded687b02aaabbabd5"} Aug 04 13:27:57.935661 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:57.935570 2580 generic.go:356] "Generic (PLEG): container finished" podID="e3c0895f-e804-4682-b6c8-44f1702e7408" containerID="9fdc2328e8e8949ae9307ab04d46e1aa0821a16a8d58d4c9b3e9b045838fee0c" exitCode=0 Aug 04 13:27:57.936057 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:57.935657 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" event={"ID":"e3c0895f-e804-4682-b6c8-44f1702e7408","Type":"ContainerDied","Data":"9fdc2328e8e8949ae9307ab04d46e1aa0821a16a8d58d4c9b3e9b045838fee0c"} Aug 04 13:27:59.079550 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:59.079524 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" Aug 04 13:27:59.150179 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:59.150137 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c0895f-e804-4682-b6c8-44f1702e7408-maas-api-service-ca\") pod \"e3c0895f-e804-4682-b6c8-44f1702e7408\" (UID: \"e3c0895f-e804-4682-b6c8-44f1702e7408\") " Aug 04 13:27:59.150515 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:59.150491 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c0895f-e804-4682-b6c8-44f1702e7408-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e3c0895f-e804-4682-b6c8-44f1702e7408" (UID: "e3c0895f-e804-4682-b6c8-44f1702e7408"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:27:59.250799 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:59.250708 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c0895f-e804-4682-b6c8-44f1702e7408-maas-api-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:27:59.949215 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:59.949182 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" Aug 04 13:27:59.949395 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:59.949223 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2" event={"ID":"e3c0895f-e804-4682-b6c8-44f1702e7408","Type":"ContainerDied","Data":"6cc30bf7735db8980ad78814b1690b09251e0e14b789d0ded687b02aaabbabd5"} Aug 04 13:27:59.949395 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:27:59.949271 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6cc30bf7735db8980ad78814b1690b09251e0e14b789d0ded687b02aaabbabd5" Aug 04 13:28:06.850618 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:06.850584 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq"] Aug 04 13:28:06.851069 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:06.850788 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" podUID="1b2479db-a867-4267-b681-f504134b34b4" containerName="maas-api" containerID="cri-o://a5f124ef75f975d3b7c7802efc94e1109f421b5276a2895362a71983f137061f" gracePeriod=30 Aug 04 13:28:06.876237 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:06.876205 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m"] Aug 04 13:28:06.880442 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:06.880416 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c1c995-29764155-v425m"] Aug 04 13:28:06.990057 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:06.990018 2580 generic.go:356] "Generic (PLEG): container finished" podID="1b2479db-a867-4267-b681-f504134b34b4" containerID="a5f124ef75f975d3b7c7802efc94e1109f421b5276a2895362a71983f137061f" exitCode=0 Aug 04 13:28:06.990253 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:06.990098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" event={"ID":"1b2479db-a867-4267-b681-f504134b34b4","Type":"ContainerDied","Data":"a5f124ef75f975d3b7c7802efc94e1109f421b5276a2895362a71983f137061f"} Aug 04 13:28:07.073004 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.072963 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2"] Aug 04 13:28:07.077254 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.077224 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c1c995-nstl2"] Aug 04 13:28:07.101303 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.101232 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:28:07.222001 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.221968 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls\") pod \"1b2479db-a867-4267-b681-f504134b34b4\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " Aug 04 13:28:07.222203 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.222027 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2479db-a867-4267-b681-f504134b34b4-openshift-service-ca\") pod \"1b2479db-a867-4267-b681-f504134b34b4\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " Aug 04 13:28:07.222203 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.222083 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnlwf\" (UniqueName: \"kubernetes.io/projected/1b2479db-a867-4267-b681-f504134b34b4-kube-api-access-cnlwf\") pod \"1b2479db-a867-4267-b681-f504134b34b4\" (UID: \"1b2479db-a867-4267-b681-f504134b34b4\") " Aug 04 13:28:07.222408 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.222386 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b2479db-a867-4267-b681-f504134b34b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b2479db-a867-4267-b681-f504134b34b4" (UID: "1b2479db-a867-4267-b681-f504134b34b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 13:28:07.224419 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.224394 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2479db-a867-4267-b681-f504134b34b4-kube-api-access-cnlwf" (OuterVolumeSpecName: "kube-api-access-cnlwf") pod "1b2479db-a867-4267-b681-f504134b34b4" (UID: "1b2479db-a867-4267-b681-f504134b34b4"). InnerVolumeSpecName "kube-api-access-cnlwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 13:28:07.224511 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.224435 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b2479db-a867-4267-b681-f504134b34b4" (UID: "1b2479db-a867-4267-b681-f504134b34b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 13:28:07.322674 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.322634 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnlwf\" (UniqueName: \"kubernetes.io/projected/1b2479db-a867-4267-b681-f504134b34b4-kube-api-access-cnlwf\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:28:07.322674 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.322668 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2479db-a867-4267-b681-f504134b34b4-maas-api-tls\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:28:07.322674 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.322677 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2479db-a867-4267-b681-f504134b34b4-openshift-service-ca\") on node \"ip-10-0-132-96.ec2.internal\" DevicePath \"\"" Aug 04 13:28:07.996872 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.996823 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" Aug 04 13:28:07.996872 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.996864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq" event={"ID":"1b2479db-a867-4267-b681-f504134b34b4","Type":"ContainerDied","Data":"097cd6866cea34818d08af16443bb0a66b88892d2ddcd8a160f06b0e69d4cdd0"} Aug 04 13:28:07.997397 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:07.996929 2580 scope.go:117] "RemoveContainer" containerID="a5f124ef75f975d3b7c7802efc94e1109f421b5276a2895362a71983f137061f" Aug 04 13:28:08.019399 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:08.019366 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq"] Aug 04 13:28:08.022182 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:08.022151 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c1c995-f48f778cb-49hcq"] Aug 04 13:28:08.277618 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:08.277541 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b2479db-a867-4267-b681-f504134b34b4" path="/var/lib/kubelet/pods/1b2479db-a867-4267-b681-f504134b34b4/volumes" Aug 04 13:28:08.277818 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:08.277806 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20fe25bd-f2dc-4af9-aac0-dfefe64cce1f" path="/var/lib/kubelet/pods/20fe25bd-f2dc-4af9-aac0-dfefe64cce1f/volumes" Aug 04 13:28:08.278084 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:08.278072 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3c0895f-e804-4682-b6c8-44f1702e7408" path="/var/lib/kubelet/pods/e3c0895f-e804-4682-b6c8-44f1702e7408/volumes" Aug 04 13:28:09.003457 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:09.003423 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8qmrl"] Aug 04 13:28:20.503196 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:20.503168 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547f69df5b-65lmk_cffd9de0-48e0-4554-a8fc-1784fdc27d7b/manager/0.log" Aug 04 13:28:20.513971 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:20.513946 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547f69df5b-65lmk_cffd9de0-48e0-4554-a8fc-1784fdc27d7b/copy-manifests/0.log" Aug 04 13:28:20.791493 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:20.791324 2580 scope.go:117] "RemoveContainer" containerID="8c37f7af96374181498f3b60cd358924e23f37c613103a029ba016e0a40b91ad" Aug 04 13:28:20.813426 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:20.800659 2580 scope.go:117] "RemoveContainer" containerID="e02f90dfbfcca524a4dfd489998f56fdd84183441175d5ff420b833f2a72318a" Aug 04 13:28:21.304442 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:21.304412 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764140-p2hzt_33a3df01-05da-4286-b1e2-7cf828b0bf3b/cleanup/0.log" Aug 04 13:28:21.410597 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:21.410573 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764155-tb5xx_78ee6716-8e41-4074-b4e2-d0322d9b75c3/cleanup/0.log" Aug 04 13:28:22.830778 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:22.830750 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-8qmrl_a99054ca-68ee-4530-b475-a7c17302681c/limitador/0.log" Aug 04 13:28:23.422661 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:23.422622 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pglv5_9f58dcf7-d354-4ae7-ac32-6f4c515e77d5/discovery/0.log" Aug 04 13:28:24.119115 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:24.119077 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ccbd485dd-85zhv_773d5742-06cb-473c-a6d8-2cd2b6beef52/router/0.log" Aug 04 13:28:24.780177 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:24.780148 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvvkps_80901ac2-7f0d-4b18-9ded-81f355bfcc78/main/0.log" Aug 04 13:28:33.598517 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:33.598481 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tztk9_3967f0e1-aa24-4602-977c-c67b09b0974f/global-pull-secret-syncer/0.log" Aug 04 13:28:33.705200 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:33.705163 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jj94z_77951ab4-2d3d-4f26-b3e7-04ad71384895/konnectivity-agent/0.log" Aug 04 13:28:33.805290 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:33.805255 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-96.ec2.internal_afdaddb70d01a23ead7adc4ab559b1de/haproxy/0.log" Aug 04 13:28:38.844552 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:38.844516 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-8qmrl_a99054ca-68ee-4530-b475-a7c17302681c/limitador/0.log" Aug 04 13:28:40.717144 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:40.717110 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-jchjx_2ea55105-3c62-470a-b30d-667698ea32a2/cluster-monitoring-operator/0.log" Aug 04 13:28:41.128314 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:41.128288 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hq7n4_35ca5d62-6868-4709-92db-1121212793fa/node-exporter/0.log" Aug 04 13:28:41.154325 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:41.154301 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hq7n4_35ca5d62-6868-4709-92db-1121212793fa/kube-rbac-proxy/0.log" Aug 04 13:28:41.181418 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:41.181390 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hq7n4_35ca5d62-6868-4709-92db-1121212793fa/init-textfile/0.log" Aug 04 13:28:41.579009 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:41.578978 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-w6dfr_cb124e3c-963d-4201-be69-3ecb41e9a37e/prometheus-operator-admission-webhook/0.log" Aug 04 13:28:41.610595 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:41.610570 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6766476b6b-s744z_51caa8d7-0b5b-413a-ad24-214e17fdcd88/telemeter-client/0.log" Aug 04 13:28:41.637850 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:41.637805 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6766476b6b-s744z_51caa8d7-0b5b-413a-ad24-214e17fdcd88/reload/0.log" Aug 04 13:28:41.662011 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:41.661986 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6766476b6b-s744z_51caa8d7-0b5b-413a-ad24-214e17fdcd88/kube-rbac-proxy/0.log" Aug 04 13:28:43.326522 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:43.326496 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-bfzn7_c8ff1675-85c7-4e20-8404-3e4968add75c/networking-console-plugin/0.log" Aug 04 13:28:44.003950 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.003910 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/1.log" Aug 04 13:28:44.008070 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.008044 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-dtb6d_63ed7430-8180-47e4-a254-6c34bcce3481/console-operator/2.log" Aug 04 13:28:44.011678 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.011653 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg"] Aug 04 13:28:44.011991 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.011978 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b2479db-a867-4267-b681-f504134b34b4" containerName="maas-api" Aug 04 13:28:44.012040 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.011993 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b2479db-a867-4267-b681-f504134b34b4" containerName="maas-api" Aug 04 13:28:44.012040 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.012010 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c0895f-e804-4682-b6c8-44f1702e7408" containerName="revoke-keys" Aug 04 13:28:44.012040 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.012016 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c0895f-e804-4682-b6c8-44f1702e7408" containerName="revoke-keys" Aug 04 13:28:44.012148 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.012070 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c0895f-e804-4682-b6c8-44f1702e7408" containerName="revoke-keys" Aug 04 13:28:44.012148 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.012081 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b2479db-a867-4267-b681-f504134b34b4" containerName="maas-api" Aug 04 13:28:44.019989 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.019967 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.026687 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.026653 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kqx2z\"/\"openshift-service-ca.crt\"" Aug 04 13:28:44.026776 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.026750 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kqx2z\"/\"kube-root-ca.crt\"" Aug 04 13:28:44.028028 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.028002 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kqx2z\"/\"default-dockercfg-6pz8c\"" Aug 04 13:28:44.030475 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.030451 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg"] Aug 04 13:28:44.147127 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.147095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-proc\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.147127 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.147132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-sys\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.147352 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.147168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-podres\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.147352 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.147251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jczd\" (UniqueName: \"kubernetes.io/projected/e9c4ae73-5885-404b-8559-a58121121e84-kube-api-access-9jczd\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.147352 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.147310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-lib-modules\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248603 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-proc\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248603 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-sys\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248903 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-podres\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248903 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248682 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-proc\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248903 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jczd\" (UniqueName: \"kubernetes.io/projected/e9c4ae73-5885-404b-8559-a58121121e84-kube-api-access-9jczd\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248903 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248746 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-sys\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248903 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-lib-modules\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248903 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248798 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-podres\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.248903 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.248886 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9c4ae73-5885-404b-8559-a58121121e84-lib-modules\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.257884 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.257806 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jczd\" (UniqueName: \"kubernetes.io/projected/e9c4ae73-5885-404b-8559-a58121121e84-kube-api-access-9jczd\") pod \"perf-node-gather-daemonset-7pwcg\" (UID: \"e9c4ae73-5885-404b-8559-a58121121e84\") " pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.330759 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.330722 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:44.466664 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.466634 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg"] Aug 04 13:28:44.468372 ip-10-0-132-96 kubenswrapper[2580]: W0804 13:28:44.468344 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode9c4ae73_5885_404b_8559_a58121121e84.slice/crio-621733ee6931d67a43f0d1c52663764dfd9d0c5661d9fa817d07a7e21b11b1c8 WatchSource:0}: Error finding container 621733ee6931d67a43f0d1c52663764dfd9d0c5661d9fa817d07a7e21b11b1c8: Status 404 returned error can't find the container with id 621733ee6931d67a43f0d1c52663764dfd9d0c5661d9fa817d07a7e21b11b1c8 Aug 04 13:28:44.741624 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:44.741595 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-6vsh7_b6ddfc52-3897-42d6-87cd-f78774091ecb/download-server/0.log" Aug 04 13:28:45.198563 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:45.198530 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" event={"ID":"e9c4ae73-5885-404b-8559-a58121121e84","Type":"ContainerStarted","Data":"6814a557d2808269f3e6243a7a1f525fb3e63c5bf9f7c10e3212c72b9f3cd305"} Aug 04 13:28:45.198751 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:45.198593 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:45.198751 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:45.198616 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" event={"ID":"e9c4ae73-5885-404b-8559-a58121121e84","Type":"ContainerStarted","Data":"621733ee6931d67a43f0d1c52663764dfd9d0c5661d9fa817d07a7e21b11b1c8"} Aug 04 13:28:45.217802 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:45.217752 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" podStartSLOduration=2.217736172 podStartE2EDuration="2.217736172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 13:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 13:28:45.215644966 +0000 UTC m=+3685.452076346" watchObservedRunningTime="2026-08-04 13:28:45.217736172 +0000 UTC m=+3685.454167542" Aug 04 13:28:45.477892 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:45.477795 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-bp4c4_5bf4ec3c-0d13-45e3-8cea-c2979f4fa689/volume-data-source-validator/0.log" Aug 04 13:28:46.667002 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:46.666960 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sj8xn_20f59554-f8ad-4737-9e31-21dc6849e377/dns/0.log" Aug 04 13:28:46.692956 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:46.692928 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sj8xn_20f59554-f8ad-4737-9e31-21dc6849e377/kube-rbac-proxy/0.log" Aug 04 13:28:46.852484 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:46.852452 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vdjr9_4ae7801d-f289-447f-8932-37938fe6924c/dns-node-resolver/0.log" Aug 04 13:28:47.672078 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:47.672037 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wwggp_43b4e662-0e13-433c-a7de-e198c6685657/node-ca/0.log" Aug 04 13:28:49.045806 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:49.045770 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pglv5_9f58dcf7-d354-4ae7-ac32-6f4c515e77d5/discovery/0.log" Aug 04 13:28:49.256634 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:49.256602 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ccbd485dd-85zhv_773d5742-06cb-473c-a6d8-2cd2b6beef52/router/0.log" Aug 04 13:28:50.128931 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:50.128895 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-69scq_0f2035c4-94d3-4ae0-b90e-71e39bafac56/serve-healthcheck-canary/0.log" Aug 04 13:28:50.683272 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:50.683232 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-j56ss_6a350f36-d1a5-462b-8f92-eea0d0a47586/insights-operator/0.log" Aug 04 13:28:50.684824 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:50.684790 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-j56ss_6a350f36-d1a5-462b-8f92-eea0d0a47586/insights-operator/1.log" Aug 04 13:28:50.707858 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:50.707805 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mx47_49832075-0827-4a88-97eb-c9653d2ba9b2/kube-rbac-proxy/0.log" Aug 04 13:28:50.734361 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:50.734333 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mx47_49832075-0827-4a88-97eb-c9653d2ba9b2/exporter/0.log" Aug 04 13:28:50.765851 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:50.765802 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mx47_49832075-0827-4a88-97eb-c9653d2ba9b2/extractor/0.log" Aug 04 13:28:51.213362 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:51.213333 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kqx2z/perf-node-gather-daemonset-7pwcg" Aug 04 13:28:53.959045 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:53.959016 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547f69df5b-65lmk_cffd9de0-48e0-4554-a8fc-1784fdc27d7b/manager/0.log" Aug 04 13:28:53.978669 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:53.978640 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547f69df5b-65lmk_cffd9de0-48e0-4554-a8fc-1784fdc27d7b/copy-manifests/0.log" Aug 04 13:28:55.102486 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:55.102448 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pglv5_9f58dcf7-d354-4ae7-ac32-6f4c515e77d5/discovery/0.log" Aug 04 13:28:55.288685 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:55.288652 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ccbd485dd-85zhv_773d5742-06cb-473c-a6d8-2cd2b6beef52/router/0.log" Aug 04 13:28:55.656569 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:55.656537 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c4cd7f74c-9xl8g_4463d891-66a6-4633-a993-f384308925ec/manager/0.log" Aug 04 13:28:55.685376 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:55.685349 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-587zc_46008ad0-353e-4e70-87bc-a35a2f7c86ee/openshift-lws-operator/0.log" Aug 04 13:28:57.074884 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:57.074853 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pglv5_9f58dcf7-d354-4ae7-ac32-6f4c515e77d5/discovery/0.log" Aug 04 13:28:57.212072 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:28:57.212040 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ccbd485dd-85zhv_773d5742-06cb-473c-a6d8-2cd2b6beef52/router/0.log" Aug 04 13:29:01.279343 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:01.279305 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-f2k9x_0f297761-617d-489b-b1f1-710bc3d8b11c/kube-storage-version-migrator-operator/1.log" Aug 04 13:29:01.281153 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:01.281123 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-f2k9x_0f297761-617d-489b-b1f1-710bc3d8b11c/kube-storage-version-migrator-operator/0.log" Aug 04 13:29:02.385896 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:02.385866 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bv8vm_960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5/kube-multus-additional-cni-plugins/0.log" Aug 04 13:29:02.411852 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:02.411792 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bv8vm_960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5/egress-router-binary-copy/0.log" Aug 04 13:29:02.440322 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:02.440293 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bv8vm_960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5/cni-plugins/0.log" Aug 04 13:29:02.467412 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:02.467384 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bv8vm_960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5/bond-cni-plugin/0.log" Aug 04 13:29:02.498976 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:02.498952 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bv8vm_960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5/routeoverride-cni/0.log" Aug 04 13:29:02.525029 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:02.524997 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bv8vm_960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5/whereabouts-cni-bincopy/0.log" Aug 04 13:29:02.550951 ip-10-0-132-96 kubenswrapper[2580]: I0804 13:29:02.550925 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bv8vm_960ba42c-e184-4c44-b6c1-f5ec2ee9e3a5/whereabouts-cni/0.log"